A lot of people don't grasp just how bad invasive animals can be.
One of the most significant problems is large snakes in Florida. At one point, Burmese pythons became popular as pets - and thus people did not realize just how big these snakes get. So, people released them, they became established, and now we're seeing drastic drops in mammal populations in the Everglades.
One common theme in science fiction is colonizing planets and introducing Terran species. A number of writers have explored the negative effect this might have on the alien world.
So, would Terran species out-compete native species? The answer is obvious: It depends. It depends on the state of the ecosystem. (And other writers have explored the results of the inability of Terran species, including humans, to compete with the natives).
Which way would it go? Here are some things to consider:
1. The age of the alien world. In general, the age of the world, at least in part, determines the complexity of the ecosystem.
2. Whether the life on the planet is based on the same amino acids as Earth life - if it is not, then colonization *requires* the destruction of the alien ecosystem *or* significant alterations to introduced Terran species - including humans. There's a story there.
3. Whether there has recently been a mass extinction event. Do humans arrive right after a major volcanic eruption, asteroid strike or other crisis? (One interesting concept to explore would be a major volcanic eruption cooling the planet, humans arrive and think that's the normal climate...)
4. How similar the gravity, day length, and atmospheric composition are to Earth. The more different they are, the more likely it is that native life will choke out introduced Terran species.
Of course, you can also tweak all of these various factors to get the result you want.