I think there are two main aspects to this that aren't really being addressed fully.
1) What are our criteria for granting civil rights?
Right now this is tied up with questions of personhood and is very poorly defined. Short of appeals to religion and souls or definitively speaking only about humans and holding us as inherently special, it is very hard to draw a clean bright line that defines who is a person and who isn't.
It's entirely possible that AIs can be defined as non-persons by default. But if we tried to construct a more nuanced version of personhood, incorporating non-human entities that showed the traits that are the reason we grant rights to people, then it becomes an open question.
As an example, at what point do we decide to grant a non-human animal the right to life? Is there a clear difference between such an animal and an AI like that in the story
Epoch (chapter 13 of that book) by Cory Doctorow, where the AI is tied to a particular structure of a particular computer, with a well defined physical existence that can be ended by shutting it down? I personally have a hard time separating those questions. If there was an AI and an animal I could interact with in similar ways I would have to offer them the same status.
2) What are we referring to as an AI?
The rule I've heard from AI researchers is that AI is anything we can't do yet. AI problems have been solved, but then they are rarely considered actual AI problems any more. No-one would consider voice recognition core to an AI any more, mostly because we are used to it as part of systems we don't exactly consider 'intelligent'.
So are there a set of aspects that we would require to call a system AI? A few possibilities;
- Pass a Turing test. But under what circumstances?
- Has an actual sense and identity of self. Except how can you tell (reduces to Turing test)? And is this even desirable (see Charles Stross's 'prosthetic conscience' that projects it's identity onto the sociopath it is guiding)?
- Conciousness. But again, how can you tell (another Turing reduction)? And is there even a clear difference between extreme calculation and concious thought?