It means stopping one recording and starting another, and defines the point in time that determines the duration of the run. It can be used to, say, automatically save the state as a checkpoint to return to after a mistake.
For segmented speedruns, you are correct, but more commonly what someone means when they say split is the speedrunner has a separate program running on his machine with a list of key points of progress in the game and the runner previous best times to that spot. When the speedrunner wants to race against his/herself, the speedrunner can start a split program with their previous record and play the game through. When a key point is reached, they press some button to denote that they reached this check point and the program will mark the time it took to reach it and tell you whether you are ahead or behind the previous record. Speedrunners use this to gauge whether they want to continue trying to beat the record or to restart and try again.