Every time this particular story is brought up by skeptics it always leads to the same conclusion that is often correct in some basic sense (i.e., 'we should not want to worship a vengeful God who -- maliciously--slaughters innocent children instead of the guilty party'), but the reasoning which leads up to the conclusion that this applies to the biblical God is flawed. In other words, their ethics are okay, but their understanding of the historical and philosophical situation is inaccurate.
And--very often--the terms they use to describe the situation are imprecise, and need more refinement and clarity. Nuances and emotional overtones can be seriously misleading.
For example, "vengeful" to a modern reader sounds like someone is consumed by and/or enslaved to a revenge-centered passion. But to the biblical participant, 'vengeance' is strictly a 'passion for true justice--both social and ethical'. And take "innocent". This word suggests that there is somehow a 'merit' issue involved--that only "guilty" people should be forced to die before they "get old". It suggests (via nuance or connotation) that there should be a linear correlation between life span and morality, when even the Bible consistently points out that the 'wicked' often live long, satisfying lives--at the expense of the righteous and/or poor. It suggests that God is somehow "obligated" to keep everyone alive to some statistical average age. Or that the countless children who have died from natural causes in infancy or in minority over the millennia were somehow 'wronged' by God--in spite of their innocence. We feel grief and violation and shock and a sense of imbalance/injustice when such occurs, but there is no solid logic for believing that a breach of ethics has occurred. God expresses grief about even judgments on the wicked when death or tragedy is inflicted on them in His judgment--even when we might not (See His weeping over His judgment on Moab in Jeremiah 48:29-33).
So, we must attempt a clarity of vocabulary and argument to avoid the extremes of either inflammatory slander or facile justifications (like referring to God as a "genocidal son-murdering prick for example).
Firstly, this wasn't a "genocide" of the Egyptians first borns.
Moses said, “Thus says the Lord: About midnight I will go out through Egypt. 5 Every firstborn in the land of Egypt shall die, from the firstborn of Pharaoh who sits on his throne to the firstborn of the female slave who is behind the handmill, and all the firstborn of the livestock. 6 Then there will be a loud cry throughout the whole land of Egypt, such as has never been or will ever be again. 7 But not a dog shall growl at any of the Israelites—not at people, not at animals—so that you may know that the Lord makes a distinction between Egypt and Israel. 8 Then all these officials of yours shall come down to me, and bow low to me, saying, ‘Leave us, you and all the people who follow you.’ After that I will leave.” And in hot anger he left Pharaoh.
Strictly speaking, God's action on that first Passover night was somehow/strangely impartial. When He judged Egypt, he somehow did something 'analogous' to Israel. This tenth plague is a last resort by God, to overthrow the Pharaoh-led (and hence, divinely-sanctioned under Egyptian theology) oppression of the Israelites. There are nine+ earlier messages to Egypt to abandon their systematic oppression (plus the additional sign given at the beginning, duplicated in some way by the Egyptian magicians).
All of the plagues/disasters/blows to Egypt were designed to subvert the moral and theological justifications of the use of Pharaonic power/authority against the Israelites. This can be seen from the texture of the plagues: they were deliberate challenges to many of the Egyptian deities (one of whom was Pharaoh himself), demonstrating their powerlessness and unreality. Even the choosing the 'firstborn' motif fits into this texture. This would also witness to the superiority of God to the Egyptian people -this was not a mere petty slight by God. As with everything He does, it had purposes that far extended the basic surface reading most do when confronted with the Genesis story.
