To imply that not looking at someone makes them invisible is silly.
Please, drop the Grammar Cop routine, and go (re-)read Anton LaVey's words on this subject.
"Invisible" means "unable to be seen".
The word doesn't specify person: if YOU are unable to see something, then it is invisible TO YOU.
Something that goes unnoticed
Something (or someone) that hides outside of your field of vision, or that creeps in your peripheral, is similarly unseen
The secret to invisibility lies precisely in being where you will not be seen or noticed
That is all there is to say on this subject in the public forums. It's a topic more fit for Downstairs.