It’s only currently possible to start one in 7 states and the District of Columbia. So avoid Colorado, Iowa, Kansas, Montana, Oklahoma, Rhode Island, Texas, New Hampshire, and Utah. Though New Hampshire only accepts them for probate as a spouse can inherit without paying taxes etc.
This is known as common law marriage and has the requirement that you _PRESENT_ as married to the world. If you don't present as married you are not common law married.
Even in the remaining 6 states you generally have to represent yourselves as married in some context like filing taxes jointly. https://en.wikipedia.org/wiki/Common-law_marriage_in_the_Uni...
PS: California does accept common law marriages formed in another state.