If I had to pin down a single group, I'd lay the best odds on the Mafia. They were extremely pissed off with Bobby Kennedy's campaign to crack down on them, and desperately wanted him out of the AG's office.

The fact that Oswald was killed by a mob enforcer is also a very strong clue.