[kamil_gr]

In any case, AGI or not, the external observer still faces a choice - algorithmic artistry or genuine subjectivity.

[Jensson]

Yeah, but it is very hard to argue if the machine figures out consciousness exists from first principles rather than from human input.

Since consciousness isn't observable the only way to realize the concept exists is to be conscious yourself, or have the concept explained to you.