[HectorRamos]

Hey,

I'm Héctor, a Parse developer advocate for the past 3.5 years. I've probably seen every question related to Parse, spend most days supporting apps with millions of users, and hopefully I can address some of these.

You bring up some valid points, some specific to Parse, others related to backend systems in general.

- "The real problems show up when you've all of a sudden got these old apps out in the field with no way to upgrade them to a newer version of the DB. If you aren't being diligent and making all your Parse DB calls through their cloud functions, you are guaranteed to end up in a situation where your older mobile clients are holding back schema and feature evolution."

Maintaining compatibility with a long tail of old, stale versions of your clients in the field is something all of us need to deal with as app developers. Our examples do place more emphasis on client-originated object updates and queries due to our approach of making it very easy for developers to get started. Some of our more sophisticated developers end up using the approach you've used as an example of moving as much of the operations over to Cloud Code. This is, in fact, one of the reasons we decided to work on our Cloud Code server-side compute environment.

- "beforeSave/afterSave triggers are completely unusable! They are unbelievably limited and each one you add slows your app down by as much time as it takes to run. If you want to do something like add users to Mailchimp, you have now tied the length of time it takes to save a user to the length of time it takes to talk to mailchimp!"

A beforeSave, by definition, will block a save until the operation has finished. Save operations on the client side are performed asynchronously, so this should generally not block your app. An afterSave can be used for things such as sending an email through Mailchimp, which in general you would not block a user operation on, but rather these would be queued up. There are some sensible default timeouts in place to discourage lengthy operations that could lead to a unsatisfying user experience (e.g. creating a new row should not take more than 3 seconds). If you are OK with having the user wait more than 3 seconds, you can use Webhooks to redirect the beforeSave/afterSave/cloud-function computation to a third party server of your choosing. In that case, you would have up to 30 seconds to return a response to the client.

- "There is absolutely no way to test your parse code other than uploading it to a parse app and hitting it. You can hack together some node code that sort-of tests it, but it's nowhere near representative of what happens on the server."

This is a big pain point many of our users have highlighted. Cloud Code is not node, so the best way to test code is by running it on Cloud Code itself. We do not have a good solution for satisfying this need at this time, so I would suggest using node.js on a third party server and taking advantage of Webhooks. This effectively allows you to Bring Your Own Server and provides you with limitless flexibility on what can be done as part of a save hook or cloud function call.

- "You can't manage Parse.Config programatically. There's a web interface and that's that. None of your config can live in your git repo because there's no way to read or write it from the command-line."

I think that everything you can do on Parse should have a REST API equivalent. We've recently launched a Hooks API as well as a Schema API. Perhaps a Config API can be considered. I personally use Config to store credentials that I would not want to be checked in to version control, and I use Parse Objects for anything that changes often enough to require programmatic updates.

- "The schema tools are pretty bad on the website. IIRC they introduced REST APIs to make changes to schema recently, but unless you plan on writing tooling around those, you'll be manually syncing schema between prod/staging at some point in your life and you'll probably also miss a critical column that got added."

You can clone an app and its entire schema from your Settings. This is the best way to ensure there is a 1 to 1 mapping between prod and staging. One thing that is missing from this is cloning data from prod to staging. Bigger apps, with gigabytes of data, could take a very long time to clone. In this case, I would suggest having some sort of seed script that populates your staging database with placeholder data. I understand this does not solve the problem of having an exact copy of your production database, but this is not a simple problem to solve (would you want to keep both apps in sync as the prod app gets updated? would you be willing to wait hours/days for your an initial sync from prod to staging?). With that said, this is something we're aware of and hope to be able to address soon. Stay tuned.

- "Parse has a bunch of weird datatypes that act strangely in their cloud code. Want to make a pointer and store it in a parse field? Construct its internal representation and set it! I wish I was kidding. If you construct a pointer incorrectly, it can actually create a phantom object in another table. Also wish I was kidding about this."

Mucking with JSON to successfully create a valid pointer sounds like an easy way to introduce bugs in your code. You shouldn't need to to do this, use YourClass.createWithoutData(objectId) instead.[1]

- "createdAt and updatedAt are special and Parse won't let you touch them or set them. Heaven forbid you want to actually set these to a meaningful time other than the actual last time the record was written. Nope. So you end up making your own updatedAt columns and just using those."

createdAt and updatedAt are automatic fields set by the underlying mongo database, and you can depend on these being accurate as no user or admin operation can force them to use arbitrary values. You're definitely welcome to create your own Date fields if your use case requires arbitrary values. This shouldn't be too much of a burden, IMHO.

Hopefully this helps! As always, if you have any questions, feel free to join us on our developer community[2].

[1]: http://parse.com/docs/js/api/symbols/Parse.Object.html#.crea...

[2]: https://groups.google.com/forum/#!forum/parse-developers

[mmastrac]

My point is mainly that Parse is antithetical to good developer practices that include 1) multiple environments 2) fully automated deployment operations in every way, and 3) testing.

- "Maintaining compatibility with a long tail of old, stale versions of your clients in the field is something all of us need to deal with as app developers. Our examples do place more emphasis on client-originated object updates and queries due to our approach of making it very easy for developers to get started. Some of our more sophisticated developers end up using the approach you've used as an example of moving as much of the operations over to Cloud Code. This is, in fact, one of the reasons we decided to work on our Cloud Code server-side compute environment."

The problem is that Parse does not offer an "afterRead" hook, so if you offer a field and a client consumes it, you must continue to offer that field until that client no longer exists. When writing traditional server APIs, I can synthesize a field for older clients during read options. With Parse, not so. Parse clients old and new read the same, raw table no matter what -- this is where I feel Parse fundamentally fails.

- "A beforeSave, by definition, will block a save until the operation has finished. Save operations on the client side are performed asynchronously, so this should generally not block your app. An afterSave can be used for things such as sending an email through Mailchimp, which in general you would not block a user operation on, but rather these would be queued up. There are some sensible default timeouts in place to discourage lengthy operations that could lead to a unsatisfying user experience (e.g. creating a new row should not take more than 3 seconds). If you are OK with having the user wait more than 3 seconds, you can use Webhooks to redirect the beforeSave/afterSave/cloud-function computation to a third party server of your choosing. In that case, you would have up to 30 seconds to return a response to the client."

If it is the case that afterSave does not block a save, then the documentation is pretty poor around this. My understanding is that afterSave and beforeSave are both required to complete before you'll get a return value from saving an object. If this is not the case, I suggest making this clearer in the docs.

- "This is a big pain point many of our users have highlighted. Cloud Code is not node, so the best way to test code is by running it on Cloud Code itself. We do not have a good solution for satisfying this need at this time, so I would suggest using node.js on a third party server and taking advantage of Webhooks. This effectively allows you to Bring Your Own Server and provides you with limitless flexibility on what can be done as part of a save hook or cloud function call."

At the point where you are using Webhooks like this, the advantages of Parse disappear entirely. You can get far more flexibility from setting up a node.js server that hits MongoLab (or some other hosted Mongo). Honestly I feel that it is absolutely _wrong_ for your DB to block on a backend server dealing with app logic. This is an inverted version of what your design should look like.

- "I think that everything you can do on Parse should have a REST API equivalent. We've recently launched a Hooks API as well as a Schema API. Perhaps a Config API can be considered. I personally use Config to store credentials that I would not want to be checked in to version control, and I use Parse Objects for anything that changes often enough to require programmatic updates."

If there's a way to set Parse.Config via REST (maybe there's an undocumented PUT?), it is not documented anywhere on the site. Since Parse.Config is public data to anyone using my app, I would certainly avoid putting credentials in there.

- "You can clone an app and its entire schema from your Settings. This is the best way to ensure there is a 1 to 1 mapping between prod and staging. One thing that is missing from this is cloning data from prod to staging. Bigger apps, with gigabytes of data, could take a very long time to clone. In this case, I would suggest having some sort of seed script that populates your staging database with placeholder data. I understand this does not solve the problem of having an exact copy of your production database, but this is not a simple problem to solve (would you want to keep both apps in sync as the prod app gets updated? would you be willing to wait hours/days for your an initial sync from prod to staging?). With that said, this is something we're aware of and hope to be able to address soon. Stay tuned."

It's not cloning a DB that's the problem -- it's programatically updating schema so that I can deploy schema changes to my staging environment and then deploy the matching set of schema changes to prod environment, automatically. This is not possible without a bunch of scripting. And, obviously, there is no easy way to mass-migrate parse objects to a new schema either.

- "Mucking with JSON to successfully create a valid pointer sounds like an easy way to introduce bugs in your code. You shouldn't need to to do this, use YourClass.createWithoutData(objectId) instead."

This needs to be better documented. Google searches bring up results where people are constructing pointers via JSON and this was indicated to be "the right way to do things". I don't have a reference off-hand to the post where this was mentioned, but it is definitely there.

- "createdAt and updatedAt are automatic fields set by the underlying mongo database, and you can depend on these being accurate as no user or admin operation can force them to use arbitrary values. You're definitely welcome to create your own Date fields if your use case requires arbitrary values. This shouldn't be too much of a burden, IMHO."

But when you combine the fact that I've got apps out in the field looking at updatedAt instead of my new field this becomes a very large burden. If I want to mass-import data into my system, I'm at the mercy of system timestamps. There are no options to fix this. These fields absolutely _should_ be under control of the admin, at least. If I have the master key, why not let me change it?

[HectorRamos]

It's not too difficult to create a new column that new clients understand, and use beforeSave to propagate updates to this column back to the older columns that earlier clients depend on. I'm not sure I understand the purpose of the "afterRead" hook and how it would help with this use case.

- "If it is the case that afterSave does not block a save, then the documentation is pretty poor around this. My understanding is that afterSave and beforeSave are both required to complete before you'll get a return value from saving an object. If this is not the case, I suggest making this clearer in the docs."

I was wrong earlier. A client will not get a response until an afterSave completes execution. What I should have pointed out is that ongoing http requests to external endpoints will not block the afterSave from returning. As such, you can still use Mailchimp in this case without worrying that a client will be blocked. Of course, you will want to avoid blocking the afterSave on the third party network request (e.g. adding on to the promise chain), which should not be a problem if this is truly a fire-and-forget operation.

- "At the point where you are using Webhooks like this, the advantages of Parse disappear entirely."

I think you're not giving the client-side SDKs enough credit here. Server side business logic is only one part of the equation. With Parse, you also get a data browser, analytics, push notifications, and probably most important of all, a database.

- "It's not cloning a DB that's the problem -- it's programatically updating schema so that I can deploy schema changes to my staging environment and then deploy the matching set of schema changes to prod environment, automatically. This is not possible without a bunch of scripting. And, obviously, there is no easy way to mass-migrate parse objects to a new schema either."

This feedback seems contrary to your earlier ask for a way to script Config variable updates. :) There's the Data Browser for UI based manipulation, there's a Schema API for programmatic manipulation, and there's a dedicated schema migration setting. You can use any of these approaches as needed, and depending on how much scripting you wish to do.

- "This needs to be better documented. Google searches bring up results where people are constructing pointers via JSON and this was indicated to be "the right way to do things""

Adding Parse objects to pointer fields is documented in all of the guides. createWithoutData is an alternate constructor that can be useful when creating pointers explicitly, but it's not the only way of creating a pointer. The link I posted is straight from the API Reference docs.

- "If I want to mass-import data into my system, I'm at the mercy of system timestamps. There are no options to fix this."

These two fields, as well as the object id, can be set to arbitrary values as part of a JSON import. Otherwise, you would not be able to import data exported from another Parse app.

[mmastrac]

- "I was wrong earlier. A client will not get a response until an afterSave completes execution. What I should have pointed out is that ongoing http requests to external endpoints will not block the afterSave from returning. As such, you can still use Mailchimp in this case without worrying that a client will be blocked. Of course, you will want to avoid blocking the afterSave on the third party network request (e.g. adding on to the promise chain), which should not be a problem if this is truly a fire-and-forget operation."

Honestly, this is part of the problem with Parse. It's a big, black box where the documentation doesn't _actually_ dig into the cases that are really important. For a big, black box the documentation really doesn't step up to where it needs to be.

So some stuff blocks afterSave, and some stuff doesn't. Does saving another object from afterSave block a save? What about the nested triggers on that other object? Why isn't that documented anywhere?

- "Adding Parse objects to pointer fields is documented in all of the guides. createWithoutData is an alternate constructor that can be useful when creating pointers explicitly, but it's not the only way of creating a pointer. The link I posted is straight from the API Reference docs."

I've never seen a reference to this, despite repeated Googling.

Open the JS developer's guide. The doc for"createWithData" says that it creates a "reference" to to an object. OK, that's a synonym for a pointer, well... sort of. Let's look at the actual method example:

  Creates a reference to a subclass of Parse.Object with the given id. This does not exist on Parse.Object, only on subclasses.

  A shortcut for:

   var Foo = Parse.Object.extend("Foo");
   var pointerToFoo = new Foo();
   pointerToFoo.id = "myObjectId";

So, there's no example of how this actually can be used to set a pointer value as a field. I'm expected to guess that this "reference" can be set to a field of type "pointer". And apparently I can create a pointer just by creating a new object? What? Where's that documented?

Instead we Google for "parse set pointer" and this page comes up:

https://www.parse.com/questions/javascript-api-set-and-creat...

"That format, {"_type":"Pointer","className":"User","objectId":"asdadsasd"} is what is considered a Pointer object. That is the recommended method"

[HectorRamos]

- "So some stuff blocks afterSave, and some stuff doesn't. Does saving another object from afterSave block a save? What about the nested triggers on that other object? Why isn't that documented anywhere?"

This has long been documented here: https://parse.com/docs/js/guide#cloud-code-resource-limits

- "I've never seen a reference to this, despite repeated Googling." ... "Instead we Google for "parse set pointer" and this page comes up:"

It sounds like you're leaning a lot on Google searches bringing up two year old questions. There's a massive archive of questions from the last 3 years, around 10,000 on my last count. Going back through every single comment and answer to make sure they're still accurate would be a huge undertaking, and a year ago we stopped using those archived questions.

I highly recommend reading through the entire documentation for the platform you're working on, as well as the API Reference for your SDK. You can find both at https://parse.com/docs. I think now is a good time to point out we're also accepting pull requests against our Docs[1]. We welcome diffs that make the docs clearer for everyone.

[1]: https://github.com/ParsePlatform/Docs

[mmastrac]

We can continue to debate the flaws in the documentation back-and-forth for ages, but there's not much point in it and that doesn't help with the glaring fundamental problems that have caused a company a lot of grief.

I appreciate your responses, but I'm done here.