Well no, cyclical references only stop the object from being deleted if it's still accessible. That's what the 'mark' part of 'mark and sweep' does - it marks all objects still accessible somehow from inside the program. Then the 'sweep' bit runs through all allocated blocks and frees the unmarked ones.
Now, if you're merely approximating garbage collection with reference counting then sure, you have a problem.
Now, if you're merely approximating garbage collection with reference counting then sure, you have a problem.