Usually, yes! I’m not sure how it works with remote hearings during Covid-19 but I imagine some courts are broadcasting live streams of the proceedings while others are invitation-only. In normal circumstances where courts are open you can just show up and sit in the gallery to observe. Most housing courts have an “eviction day” once a week. If you call the courthouse ahead of time the clerk can tell you what day of the week is eviction day.
If you yourself are going to be involved in a case it's a good idea to sit in a few trials so you get an idea of how things go.