Interestingly the only differences between Apoe4 and a neutral, very common variant called ApoE3 is one amino acid at position 112. The small variance appears to account for most of the negative effect of the ApoE4 genotype.
https://www.ncbi.nlm.nih.gov/pubmed/28533891