It's easy to imagine that scaling up a neural network made up of actual human neurons will quickly achieve something resembling "intelligence" and "qualia" since it's made of the same parts that presumably produce those phenomenon in us. The ethical implications are much harder to deny. Though I am sure it could be attributed to some failure of empathy as you've implied.

Perhaps people fear this leads to a future where we have technology that interacts meaningfully with existing wetware (cybernetically enhanced animals, including people).