[Seldaek]

For those looking for a technical explanation, the PHP garbage collector in this case is probably wasting a ton of CPU cycles trying to collect thousands of objects (a LOT of objects are created to represent all the inter-package rules when solving dependencies) during the solving process. It keeps trying and trying as objects are allocated and it can not collect anything but still has to check them all every time it triggers.

Disabling GC just kills the advanced GC but leaves the basic reference counting approach to freeing memory, so Composer can keep trucking without using much more memory as the GC wasn't really collecting anything. The memory reduction many people report is rather due to some other improvements we have made yesterday.

As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue. In most cases though this isn't an issue and I would NOT recommend everyone disables GC on their project :) GC is very useful in many cases especially long running workers, but the Composer solver falls out of the use cases it's made for.

[sillysaurus3]

As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue.

That sounds like a bug in the profiler, not with Composer. Observing internal time is pretty important for any profiler.

[Seldaek]

Yes it is definitely a failure of the tooling and I hear it is actually being worked on.

[haaaaaaryf]

Couldn't I also speed up Composer by using Toran Proxy? ( https://toranproxy.com/ )

[Seldaek]

That would help speed up the network part of the install/update yes. The linked patch on the other hand hopefully gets the CPU part to a decent level for most people.

[icehawk219]

Out of curiosity what tools do you use for profiling and finding these sorts of things? Plain old xdebug, xhprof, or other things? I'm going to have to jump into debugging a fairly large Symfony application within the next couple months and am on the look out for good tools to help me along.

[Seldaek]

Traditionally xdebug/xhprof were pretty decent, but xhprof has been a bit abandoned since facebook uses HHVM now.

There are two new commercial condenters though that came out in the last few months: Blackfire.io and QafooLabs.com

Both have announced support for showing GC time in profiles as a result of today's noise :) https://twitter.com/beberlei/status/539816149303955456 https://twitter.com/symfony_en/status/539815082881187841

[jexus2]

Xhprof is maintained by Phacility. They did pick up on this[1], but it's not as easy as one might think.

[1] https://secure.phabricator.com/T6675

[Seldaek]

Thanks for the correction, I wasn't aware that was part of what phabricator extracted out of facebook :)

[icehawk219]

Thanks for the info, those two will be very helpful.

[munificent]

> It keeps trying and trying as objects are allocated

Worse. GC gets triggered just by assigning references, allocation isn't even needed.