[RC_ITR]

I think it's interesting that everyone's immediate reaction now-a-days is to assume incompetence or maliciousness, rather than curiosity at the root cause (very telling this attitude has even permeated a forum for supposed 'hackers').

A high-level is that 80% of the economy is very easy to track b/c it's not very volatile (teachers, for example).

What we have seen is a huge surge in unpredictability in the most volatile 20% of jobs (mining, manufacturing, retail, etc.). The BLS can't really change their methods to catch up with this change for classic backwards compatibility and tech debt reasons.

Part of the reason 'being a quant' is so hot right now is that we truly are in weird times where volatility is much higher than most people realize across sectors of the economy (i.e. AI is changing formerly rock-solid SWE employment trends, tariffs/electricity are quickly and randomly changing domestic manufacturing profitability, etc.). This means that if you can build systems that track data better than the old official systems, you can make some decent money investing against your knowledge.

I think this is a bad state of affairs, but I don't have a good solution. Any private company won't release their data b/c it's too valuable and I am reluctant to encourage the BLS to rip up their methods when backwards compatibility is a feature worth saving.

[tonyedgecombe]

Is there really more volatility? My gut feeling is that government interventions have flattened it over recent decades. I’d like to see some real figures on this.

[RC_ITR]

https://fred.stlouisfed.org/graph/?g=1Mc3z

Manufacturing and mining are becoming much less correlated to the overall jobs market (likely, as you point out, b/c the government smooths the other sectors).

https://fred.stlouisfed.org/graph/?g=1Mc3I

This is despite being a relatively flat % of employment since 2010 (after a long period of decline).

https://fred.stlouisfed.org/graph/?g=1Mc4f

As mentioned, there is also the weirdness of SWE's going from 'better than the overall market' to 'worse than the overall market'.

https://fred.stlouisfed.org/graph/?g=1Mcer

Retail employment is also dislocating.

Those are just the examples I can think of with no research, I'm sure there are others.

[chrisco255]

Can you actually prove volatility is higher now than in the past? There have been plenty of volatile changes in the workforce over the past several decades, this is not anything new to the job market.

[logifail]

> interesting that everyone's immediate reaction now-a-days is to assume incompetence or maliciousness, rather than curiosity at the root cause

I came across this claim last week regarding recent US jobs figures:

> "All jobs gains were part time. Full-time jobs: -357K. Part-time jobs: +597K"

If this claim is true, and I have no means to tell if it is, then - regardless of one's view on whoever is in power right now - do we really expect any elected representatives to be brave enough to say that out loud at a press conference?

I don't :/

[riazrizvi]

Explain to me please why job numbers aren’t simply a matter of querying the Federal social security database? A longstanding process of polling businesses for what they want to report, followed by corrections up to one year later, has got to be a pantomime to fudge the numbers.

[tzs]

They survey businesses because the Social Security database has too much lag and does not contain enough detail.

The lag is because it is based on employer submissions that are quarterly or annual.

[riazrizvi]

Does that pass the basic common sense smell test? Everyone can see on their paycheck the amount, that is paid 30 days after any work day in the worst case. These payments are sent to a single federal bank account, and data-wise are combined with Social Security ID, sending bank id, date. It’s a bank, there’s a database. We are talking at most about 200mm records, a raspberry pi can process that query in minutes. If we can’t query this easily it’s by design. Or we could do some backflips and somersaults to try to come up with a reason for why the bureaucracy has to be more complicated.

[tzs]

The payments are deposited monthly or semiweekly (for employers with large payroll) but that's a lump sum. If you are looking at that from the government side all you can tell is whether total payroll has gone up or down. That won't tell if any change is due to a change in number of employees or a change in pay rates or some combination of that.

It isn't until the employer files their quarterly Form 941 that you'd see employment numbers. Form 941 includes the number of employees and total wages and withholding.

It isn't until the annual W-2 filings that you would see a breakdown that includes number of employees and the individual pay.

[riazrizvi]

Ah okay, this is why then. So all my other comments complaining about the lack of timeliness have this simple explanation. TIL

[RC_ITR]

Not all 'normal income' is from a "job" as we think of it and assuming that does not even come close to passing any informed person's smell test.

Parsing tax or SS payments for what a "job" is would be a logistical nightmare, because that's not what the system is designed for (unlike the BLS's system, which is designed to count jobs).

[riazrizvi]

When ppl want job numbers they want a reliable proxy for the state of the economy. Fixing it on changes to payroll-based social security payments would be far better than what we have now, if timely.

[RC_ITR]

Sure, that's personal income and can be found here:

https://fred.stlouisfed.org/series/PINCOME

[mannyv]

And yet, SS contributions are done every pay period.

Who has that data then? Treasury?

[riazrizvi]

So the answer is payments per social security id are not reported to the social security Electronic Federal Tax Payment System (EFTPS), employers only report aggregate payments. And workers and employers only report payments by individual in W2’s in January.

[chrisco255]

Probably the only reason is because the BLS and SSA are completely separate, and SSA is probably antiquated and doesn't attempt to tag or organize their data along the same parameters as whatever the BLS defines. It likely neither has the staffing nor resources to provide those hooks and realtime anonymous aggregated data for other departments to consume.

[SantalBlush]

A lot of people don't understand that collecting data is actually expensive and difficult when it doesn't involve surreptitiously stealing it via some piece of tech.

[gdulli]

Are 'hackers' allowed to have priors regarding incompetence or malice, or are we supposed to look at everything with a clean slate and no context?