My verdict this time: I found TripAdvisor slightly better than Kayak and Bing Travel.
Four years ago, I recommended travel-focused search engines, particularly Kayak, as a one-stop shop for airline deals. These services check multiple sites at once and let you book directly through the airlines. By contrast, online travel agencies such as Orbitz and Expedia did the booking for you - and tacked on several dollars in service fees.
I'm pleased to say that those fees are largely gone, partly a response to rising competition. Yet after trying all the sites again, I am still sticking with the search engines for their ease of use. Sometimes the search engines still refer me to an online travel agency for the best deal, but at least they've checked around for me.
And I can more easily use the search engines to narrow my choices based on departure times, frequent-flier programs and other criteria.
Before I get into why TripAdvisor was slightly better than Kayak and Bing Travel, I'll note that they all work similarly. Start by typing where you are leaving from and where you are going. Then pick the departure and return dates and hit "search" or "find flights."
You'll get a list of flight options, which you can sort by price and other criteria. If you prefer non-stop flights, you can uncheck boxes for one or more stops. If you need to get there by noon, you can adjust an on-screen slider to eliminate afternoon and evening flights.
Kayak and TripAdvisor let you filter out airlines that aren't part of specific frequent-flier alliances, so you can be sure you'll earn miles. Kayak also lets you choose whether you want only flights with Wi-Fi wireless service.
Let me step back to say that no search engine is complete. It all depends on deals a search engine makes with the airlines and travel agencies. Kayak and Bing, for instance, can incorporate fares from Orbitz, but not Expedia, while TripAdvisor can pull fares from Expedia, but not Orbitz. None of the ones tested includes Southwest Airlines fares.
As a workaround, these search engines do offer easy ways to search those excluded sites in a separate window. Bing also includes Southwest flight schedules in its results - just not the fares.
Based on all this so far, Kayak would win because it offers more filtering choices than the others. But that conclusion would miss nifty features elsewhere.
Bing, run by Microsoft Corp., has Visual Search Galleries that help identify the best options based on whether you're looking for a family vacation or a honeymoon destination, what time of year you want to travel and how long you are willing to sit on an airplane.
Bing also lets you search for multiple destinations at once, not just multiple airports in a region. If you're being indecisive about whether to fly to Chicago, Miami or Boston this summer, plug in all three and let Bing pull up all the options.
Charts and graphs help compare fares by departure date and length of stay. They are based on what Bing calls "observation queries" - daily checks of millions of flight and date combinations, all analyzed and stored in a database awaiting your query. By contrast, other sites that recommend the cheapest travel dates generally use data from what their users recently searched, so the results are typically incomplete.
Once you've settled on where and when to go, Bing can predict whether fares are likely to go up or down if you wait. It's based on the Farecast technology that Microsoft bought in 2008.
I tested it by performing the same dozen flight searches seven times over a week. Had I followed Bing's recommendation to buy or wait on Day One, I would have spent a total of $210 more on four flights but saved a combined $400 on seven others (The price on the 12th was unchanged). Not bad.
That said, Bing didn't always produce the lowest fares. That honor goes to TripAdvisor, based on test searches for a half-dozen different trips. TripAdvisor once even found a lower fare on Expedia than Expedia's own site did.
Three of the travel agency sites - Orbitz, Travelocity and Priceline - sometimes appeared to have a better fare by rounding down, but basic math calls for rounding $169.70 to $170 - not $169. I discovered that only by clicking on the fare - in Priceline's case, after entering my name and selecting a seat as well. TripAdvisor didn't try to hide that dollar.
I also liked TripAdvisor's tool for calculating bag fees and other add-ons. Simply enter the number of bags you are checking, whether you are have an elite frequent-flier status (which sometimes comes with free bag checking) and whether you'd like to buy food, alcohol or headphones on the flight.
TripAdvisor estimates and adds the fees to your base fare for comparison. A JetBlue flight might suddenly come out as cheapest because it lets you check one bag for free.
Kayak has such a feature, too, but it's less versatile and harder to find.
Some travelers may prefer the comfort of travel agencies such as Orbitz, which sometimes offer rebates when fares go down and send text messages informing you of delays and gate changes. These travel agencies also offer discounted flight and hotel packages. You might also get good deals by signing up for fare alerts and Twitter feeds that airlines, AirfareWatchdog.com and many other sites offer. Hotwire also offers cut-rate prices - but you might have to buy before you find out the airline and flight time.
Nonetheless, the travel search engines remain a good starting point. I've long been a fan of Kayak, but this test has opened my eyes to other options out there. I like Bing's tools for helping me decide where and when to go - as well as when to buy. But once I've made that decision, nothing beats saving money. TripAdvisor came through.