Maybe. I've heard a couple people with similar diets wonder why their plasma cholesterol is so high, whether or not it's from the dietary cholesterol. YMMV.
Your anecdotes don't match the research. Consumption of animal products is not much correlated to serum cholesterol. Serum cholesterol in turn is not much correlated to mortality. If anything it's been found that low levels are of greater concern than high levels.
Cholesterol is not really worth measuring. Triglycerides and insulin levels/sensitivity are maybe worth measuring.
Avoiding egg yolks is stupid. All the vitamins and minerals are in the yolk.
http://www.extension.iastate.edu/Publications/NCR332.pdf