Hacker News new | ask | show | jobs
by stickfigure 5284 days ago
This is something that a lot of GAE developers misunderstand: put()ing a datastore entity is not a single write operation. There are indexes to update - in your case, lots of them - and updating these indexes can require several write operations. A simple delete is one write per index but changing a value can be two operations; one to delete the old index value and one to write the new one. And since each property has two indexes (ascending and descending), these numbers are X2.

If you create your own bulk delete method, you will find that it takes exactly as many write ops as the admin console tool.

You probably have defined more indexes on your entities than you need to - you will likely be able to make your app cheaper by removing unnecessary indexes. Managing indexes carefully is a critical part of making apps affordable on GAE.

1 comments

I had "vaccumed" all indices referencing those entities before issuing a delete. Albeit, there was only once index per purged entity type. So this would not explain the 20x write operations.

Also, note that the deletions were through the "Datasore Admin" app, which was recently added. It is different from the classic Datastore Viewer.

You misunderstand how GAE indexes work.

There are two kinds of indexes:

* multi-property indexes which you configure via datastore-indexes.xml (or yaml). You can remove these by removing them from the xml/yaml and vacuuming.

* single-property indexes, which you decide when you define your data model. You can't vacuum these, and they are defined on a per-entity basis. The only way to make them go away is to re-save the relevant entities without the index defined. Note: multiproperty indexes require single-property indexes on all the properties covered.

These single-property indexes are almost certainly causing your high write op counts. You really should examine your data model with this new understanding; by removing unnecessary single-property indexes, you may be able to dramatically reduce your bill.