The stones to which you are referring, and larger that are found in and around other structures, are too large to have been cut with the method that you suggest.
I'm not sure which part you think is impossible, but it's absolutely possible to chisel a stone out of a quarry with stone tools. There were subsequent steps to work that rough shape down into the final product, of which abrasion was one of the last steps. None of them are especially sensitive to the size of the stone. We've found the rocks they used to do it littering known quarries, on obelisks and the like.