Well, no. Some common plastics like polycarbonate aren't biodegradable, and will basically never break down without application of significant heat/water/enzymatic activity/etc. For some of these, the half-life could be a great deal more even than 10s of millenia: