why are transaction costs so high? I thought one of the selling points was that the transaction cost is negligible because miners main incentive comes from mining new bitcoins?
That used to be true, until the network started hitting the "block size limit". That's the combination of two design features:
1) The mining difficulty automatically adjusts to keep the rate of new blocks down to about 1 every 10 minutes.
2) There has always been a 1 MB limit on how large a block can be.
For a long time, #2 didn't matter, because all blocks were much smaller than 1 MB. But about a year ago, transaction volumes finally rose enough that all blocks since then have been at the max. That means that rather than just taking up marginal bandwidth and disk space, transactions are directly competing with each other for the limited space in each block. That's caused transaction fees to skyrocket.
The question of whether the max block size should be raised has been extremely controversial in the bitcoin world, and the "bitcoin cash" fork was primarily focused on this question.
Transactions take up space on the chain, and the chain only writes at about 1.6 kBps (1MiB every 10 minutes). People therefore must compete on fees to be included on the chain.
Because there is not enough of space for transactions in mined blocks, you can't fit all which means people that pay more will have higher chance to get their transaction into the block.
No. The amount of miners isn't tied to how many bitcoins exist, or how many transactions are happening.
Maybe you're thinking of how when more people are mining, it becomes harder to mine. This is because there is a global fixed rate of Bitcoin creation. Every 10 minutes, 12.5 new bitcoins come into existence. Miners are effectively competing for the chance to be the one to receive those new bitcoins. If mining becomes unprofitable, then some miners will drop out, causing it to become more profitable for those remaining. Miners don't set prices or rules; price affects profitability which either causes more miners to mine or some to drop out.
There's a finite number of bitcoins, fewer and fewer bitcoins are being produced over time by design. There can only be 21 million bitcoins, and the majority have already been mined. So the reward just for completing a block (not including the transaction fees) has been steadily decreasing, and that's not even taking into account that more and more miners means it's harder to get that reward. Thus the trend is going to be towards increased transaction fees.
No, transaction volume and congestion is the thing that drives transaction fees up.
If transaction volume (and fees) stayed constant, then as the minting rate goes down, there will be a lower reward for blocks, and some miners will drop out of mining because it's not profitable for them. Miners dropping out will lower the mining difficulty, so the remaining miners will get a bigger cut. The system finds an equilibrium. Mining profitability only affects the mining difficulty and miners themselves (and the amount of resources necessary for a 51% attack, but as long as that remains obscenely high then the specifics don't really matter in this discussion).
... so bitcoin is doomed as a real currency as the prices are only going to get higher and people are not going to pay a $5 transaction fee for a $10 product.
How is the bitcoin community rationalizing this seemingly inescapable mathematical failure?
Scalability and de facto centralization was always going to be the slow death of Bitcoin,from the size of the blockchain, ASIC miners (which locked out people who couldn't afford to build warehouses of them, creating a centralized market of people who can actually mine coins) and transaction fee. Its too soon now to say that Bitcoin is dying but it certainly will at the current rate, when? Who knows but it's got pretty severe design flaws that I think weren't properly considered in development.
The transaction fees won't grow forever. Generally speaking, they just need to be high enough that miners can turn a profit. What that amount is, I don't know.
And the fact that there's a limited number of transactions that can be processed in a single block, and competition between miners to validate transactions as quickly as possible, complicates things.
What keeps the transaction fees from growing forever? If only a certain amount of transactions can be processed when a coin is mined then that means that competition will only grow as bitcoin becomes more widely used. This competition will raise prices yes? Because if you don't have a high bid then you won't get included when the coin is mined. And since coin mining becomes slower and slower over time that means that less windows of opportunity exist for even greater numbers of transactions. It certainly seems like this issue will only get worse unless there's some other factor I don't know about.
That's not really the answer. The answer is that there are more transactions being broadcast than there is space in the blocks to accommodate them. Thus creating fee pressure.
1) The mining difficulty automatically adjusts to keep the rate of new blocks down to about 1 every 10 minutes.
2) There has always been a 1 MB limit on how large a block can be.
For a long time, #2 didn't matter, because all blocks were much smaller than 1 MB. But about a year ago, transaction volumes finally rose enough that all blocks since then have been at the max. That means that rather than just taking up marginal bandwidth and disk space, transactions are directly competing with each other for the limited space in each block. That's caused transaction fees to skyrocket.
The question of whether the max block size should be raised has been extremely controversial in the bitcoin world, and the "bitcoin cash" fork was primarily focused on this question.