[justinpombrio]

Off topic, but since I worked at Kayak let me expand a bit on its relationship to ITA as I understood it.

Airlines and hotels had data about flights and available rooms. Kayak needed this data to show it in your search. This could happen in a few ways:

1. The airline or hotel could send Kayak data that had a particular format, and Kayak would deal with it. A common format, especially for hotels IIRC, was invalid csv files.

2. The airline could send its flight data to ITA, which standardized it across all airlines that used the service. ITA was a separate company from Kayak and Google flights and other flight search engines; it aggregated data but was not a search engine. [EDIT: was not primarily a search engine. See comment below.] Kayak would buy data from ITA.

3. The airline or hotel could decline to let Kayak list its flights/rooms.

Since then Google bought ITA, breaking the [EDIT: partial] separation between the most common flight-data aggregator and flight search engines. (Which was probably good for competition; I was a little concerned about the merger.) And Kayak was bought out too, by one of its competitors.

Take all this with a grain of salt: this is what I remember from years ago, and it wasn't really my area. And I certainly don't speak for Kayak.

[cracauer]

Not to mention almost all US based carriers and many others use ITA's flight search service to power their own (the carriers') websites. They can't search in the data they created, so they have to pay ITA.

The data is complicated and many-sourced. Fares and rules come out of ATPCO. Flights are separate from a different supplier. Seat availability is a massive collection of data morphing at 10 Hz, and it has nothing to do with whether seats are available. Then there is taxes and government fees which I think are now ATPCO, too. The timezone file I have seen at ITA still beats any other timezone collection I have ever seen. It's busy over there and the wires are glowing.

[bogomipz]

>"Not to mention almost all US based carriers and many others use ITA's flight search service to power their own (the carriers') websites. They can't search in the data they created, so they have to pay ITA."

Can you say why aren't carriers able to search in the data they created without using ITA?

[cracauer]

Exactly.

You have to realize the difference between a "pricing search" and a "low fare search", to use ITA terminology.

A "pricing search" knows which specific flights it goes through and you just find the fares (keep in mind the carriers file like 600 fares for planes with 20 seats). The airlines could always do that. Before ITA came along in 2001 or so that is what you were waiting on in the travel agency while the agent tries different flight combinations by hand.

"Low fare search" is also finding the flights. The carriers could not generally do that without ITA. There are exceptions such as Southwest which had a fare structure more like buses that kept the combinatorics down.

But in addition to the immense amounts of fare and rule data that pile on your if you search 400 itineraries out * 400 back you also have the seat availability system. No carrier had (probably has) a system that can answer the amount of "seat" queries that a single low fare search causes.

Keep in mind "seat availability data" has nothing to do with what seats are available. It is a high-frequency (10 Hz or whereabouts) way to turn fares on and off.

[bogomipz]

Interesting. Thanks for the reply.

>"A "pricing search" knows which specific flights it goes through and you just find the fares (keep in mind the carriers file like 600 fares for planes with 20 seats)."

Wow why so many? This seems excessive?

Could you elaborate on:

>"But in addition to the immense amounts of fare and rule data that pile on your if you search 400 itineraries out * 400 back you also have the seat availability system. No carrier had (probably has) a system that can answer the amount of "seat" queries that a single low fare search causes."

I didn't understand the "400 itineraries out * 400 back you also have the seat availability system" part. I din't understand the math and how it enables a "seat availability" system.

Lastly are all of the limitation really due the legacy Sabre booking system that the carriers all use? I guess I didn't realize what a huge innovation ITA was.

[cracauer]

I would say the lack of limitations is the problem.

Quoting from my notes for my 2006 conference talk:

The Easter Island west of Chile has - One flight departing today, one flight arriving today. Friday there was no flight at all. - fares touching Easter Island covering this flight: 1432 published, 160,000 constructed

Apparently I didn't count fare-by-rule fares. Yes they have 2 mechanisms to make up fares during the query.

The rules are complicated. In fact Turing-complete. So you can have unlimited number of fares, too.

"400x400 itineraries" means you construct 400 ways to get to the destination and 400 ways to get back, for a round-trip which is what most people search for. So if you go Boston-Hamburg all the intermediate airports cause variants. The same flight at a different time is different and causes a new itinerary. So this is really not as deep a search as you would think.

When I abused my testing rack at ITA to search for flights home (Hamburg), I used 3500x3500 itineraries (using a specially compiled QPX with limits raised). Rarely but occasionally it would find obscure flights that would make it cheaper, flights not covered by regular limits.

[cracauer]

Oh and the carriers all insist on using a specific currency conversion table for the day. We had a whole bunch of them. Nobody really defines currency rates. And there are a lot of them.

I already mentioned several hundred of timezones. IIRC all carriers were happy with the same timezone file, amazingly. I think that was because ITA was the first to do one that comprehensively?

[cracauer]

And that's just the fares. We didn't talk yet about seat availability.

You have to ask for seat availability for the complete fare-component, the thing that a fare covers. So if you go Boston-Copenhagen-Hamburg you don't just do seat queries for Bos->CPH and CPH->BOS, you are required to ask those questions in a context that says you travel BOS->HAM.

The seat availability for the same seats on the same flights on the BOS->CPH leg is different depending on whether you ask in a BOS->CPH context or a BOS->HAM context.

You can see how asking all those seat availability queries for all the different ways to fly in the query explodes nicely. Combinatorics at work.

[cracauer]

Here is more info: http://www.ai.mit.edu/courses/6.034f/psets/ps1/airtravel.pdf

It's from 2003, thing got a lot worse since then. The carriers make the rule mechanisms without having mathematicians or computer scientists at hand that watch the combinatorics. They have enough trouble not misfiling fares altogether (aka $0.02 fare NYC->London next Friday).

[cracauer]

Another example from my 2006 conference notes. Assume you want to go from Easter Island to Aalborg, Denmark. Not a commonly traveled market. Cough. Just the number of fares filed by the data directly for this route is 57. Direct fares (for calculation without considering using 2 fares) from Easter Island to Copenhagen is 157.

You can see that when itineraries between other locations want to try out going through Denmark they pull in a lot of those fares. They add up if you do 400x400 itineraries.

[snazz]

Interesting! I wonder why ITA never offered a search engine themselves, if they had all the data aggregated. And boy do I love dealing with invalid CSVs... (especially ones that contain commas within the fields by accident)

[wbl]

They did! Matrix.itasoftware.com

[setr]

I want it noted that this thing is really neat — you can provide parameters like min layover of 2 hrs, or 2 weeks, and even search only within a particular airline alliance

[snazz]

Oh, nice. It looks pretty fancy, too. I’m surprised that Google didn’t axe it in favor of Google Flights.

[cracauer]

There are some feature in matrix that flights didn't pick up. The original matrix site was even more powerful and exposed some operator-ishness, depending on what kind of account you were granted.

[justinpombrio]

I suspect that branding is powerful, so it's better to keep an existing flight search running than to axe it in favor of something else and loose all the existing customers. Google bought ITA and kept matrix.itasoftware.com running with (presumably) the same branding; Booking Holdings bought Kayak and kept kayak.com running with the same branding.

[lkirk]

The title of the site is "Matrix - ITA Software by Google". Also, on the site, they link themselves:

  Want to explore flights and get fast results?
  Try Google Flights

Almost seems like a deprecation notice...