Automation Challenges: The Tech View on the Ticketmaster Failure

We address the Taylor Swift Ticketmaster meltdown from a tech perspective, unpacking how bots can be both problems and solutions and how automation can help prevent a similar disaster.

She can make the whole place shimmer, and she can make the whole site break. On November 15, 2022, millions of fans entered Ticketmaster’s online queue for the Eras Tour, Taylor Swift’s first headlining concert tour since 2018. Fans spent hours waiting for tickets, while Ticketmaster, overwhelmed by the volume of users, paused the queue several times and delayed ticket sales for hours before finally shutting down the sale altogether.

By the time Ticketmaster stopped the presale, it had already sold approximately 97 percent of tickets for the Eras tour — that’s 2 million tickets, the highest number of tickets sold for a single artist in history. While 1.5 million fans received access to the November 15 “presale” of Eras tickets, 14 million users, including bots, swarmed Ticketmaster all at once.

In its apology blog post, Ticketmaster emphasized the amount of web traffic flooding its site, “Based on the volume of traffic to our site, Taylor would need to perform over 900 stadium shows…that’s a stadium show every single night for the next 2.5 years.”

Ticketmaster’s meltdown sparked a barrage of tweets, posts, and TikToks from Swift fans, as well as an ominous warning from Congress. Last month, members of the Senate antitrust committee met with executives of LiveNation, Ticketmaster’s parent company. How Ticketmaster prepared for Taylor Swift’s presale wasn’t the topic of conversation — monopolies were. LiveNation controls roughly 70 percent of ticket sales in the events industry, and members of Congress questioned whether Ticketmaster had something to gain from the bot-driven Eras presale chaos.

Marketing to the Masses

In August 2022, “All Too Well (10 Minute Version) (Taylor’s Version)” won a VMA for video of the year. Taylor ended her acceptance speech by announcing her new album, “Midnights”:

“I had sort of made up my mind that if you were going to be this generous and give us this [award], I thought it might be a fun moment to tell you that my brand-new album comes out October 21. And I will tell you more at midnight.”

An accompanying Instagram post soon followed, Swift describing “Midnights” as a collection of “13 sleepless nights scattered throughout my life.” From then on, Taylor’s marketing campaign swept the Internet, encouraging fans to experience their own sleepless nights.

In late September, Swift began “Midnights Mayhem with Me,” a series of TikToks set to snappy seventies game show tunes. Every few days, always at midnight sharp, Swift released a TikTok in which she would draw a numbered ball from a bingo cage and announce the name of the track that corresponded to the number. The TikToks, later posted to her Instagram story, drew massive attention, with most TikToks blowing past the 10 million viewer mark within a day. The night before her album’s release, Taylor reached millions of people outside her fanbase (affectionately known as Swifties) by releasing a teaser trailer of “Midnights” music videos during the third quarter of Thursday Night Football.

Then, only ten days after the release of record-breaking “Midnights,” whispers of a tour appeared on Twitter. Swifties spent Halloween leaking various combinations of potential venues and dates before Swift herself announced the Eras Tour, a “journey through the musical eras of [her] career (past and present!),” a day later. Several factors beyond the album’s extensive marketing campaign foreshadowed that this tour would have unprecedented demand.

It’s no surprise that more people want to purchase concert tickets after the pandemic, but Swifties are especially invested in the Eras Tour. Lover Fest, a worldwide tour of Swift’s 2019 “Lover” album, was slated to take place during the summer of 2020. Swift postponted the tour, then canceled in early 2021, giving all fans refunds.

Since the would-have-been Lover Fest, Swift has released three new albums, “folklore,” “evermore” and “Midnights,” and has re-recorded two of her most famous albums, “Fearless (Taylor’s Version)” and “Red (Taylor’s Version).” All of these details — a large and extremely loyal fanbase, an exceptional amount of new music, a well-executed marketing campaign, and a tour that will speak to every “era” of Taylor’s career — contributed to the unprecedented demand for Eras tickets.

To purchase tickets, fans were required to register with Ticketmaster’s Verified Fan program, an effort to ensure that real people, not bots, would log on to purchase tickets. On the day Swift announced the Eras Tour, over 3.5 million people registered for the Eras presale — the largest registration in history — but during the first few hours of registration, Ticketmaster lagged. And glitched. And paused.

The demand was there — so much, in fact, that Swift added seventeen extra shows to accommodate these large registration numbers. And, Ticketmaster’s site faltered under the number of eager fans long before the day of the presale.

The Tech and Automation Challenges

Swifties anticipated competition amongst themselves, but they didn’t predict their biggest competition: bots, software designed to carry out task-oriented actions by mimicking how we click, type and interact with interfaces. While bots sometimes have a negative connotation, they are like any other tool. Developers can program bots to surf the Internet for information, identify specific services and report data to people who can use it to evaluate options and make decisions.

Approximately 40 percent of Internet traffic is bots, and most of us interact with them daily, whether using a chatbot to make an online return or asking Alexa to play our favorite song. People frequently use bots in the ticket sales industry, sweeping the Internet for information on ticket inventory and pricing to determine how to get the best deal.

Put bots in non-ethical hands, though, and things can get tricky. By programming bots to create hundreds of new accounts, each of which can purchase six tickets or hack into real people’s accounts by inputting easy-to-guess passwords, people can use bots to buy highly sought-after tickets in bulk.

Some bots can bring in hundreds of tickets in less than a minute, allowing the people who programmed them to resell those tickets at a marked-up price. Additionally, bots seek out the best seats, trying to scrape up VIP tickets or front-row seats while leaving the back of the stadium untouched. The more demand there is for an event, the more likely an army of bots is at the ready.

Thinking Back: How Ticketmaster Could Have Anticipated these Automation Challenges

We’ve examined the dynamics of the flood and demand for Taylor Swift tickets from a marketing and tech perspective. Now, let’s look at the dam, the architecture of Ticketmaster’s website.

Unlike many Swifties, Ticketmaster knew bots would be an issue during the Eras Tour presale. To combat unwanted bot traffic, Ticketmaster offered their Verified Fan service to Taylor Swift. Artists who opt to use Verified Fan want to limit, if not eliminate, buyers who use bots to resell tickets for profit. By requiring people to register themselves as a “verified fan” in advance of the day of ticket sales, Ticketmaster has the opportunity to “verify” registration requests and filter out the bots from the fans, so on the day of ticket sales, wait times are shorter and the only people waiting are real people.

As another bot-preventing measure, Ticketmaster uses CAPTCHAs before a fan can purchase tickets. A CAPTCHA is an automated test designed to determine whether you are doing something or whether a bot pretending to be you is attempting to access your account. CAPTCHAs often appear when you log into an account you haven’t used in a while or when attempting to purchase something on a website.

This test can either be a random assortment of distorted letters and numbers you must type into a text box or a grid of images, accompanied with an instruction like “select all images with crosswalks.” Bots cannot identify distortions or evaluate images like people can, so while Verified Fan registrations help weed out bots on the front end, CAPTCHAs work on the back end as a final bot buffer.

Despite these efforts, millions of bots could enter the Taylor Swift presale. What went wrong?

Demand

While Ticketmaster recognized the level of ticket demand, they didn’t anticipate the extent to which demand would drive enormous bot presence. Experienced resellers would have recognized the combination of a loyal fanbase, a successful marketing campaign, and an extended time away from touring. These resellers would have inferred that fans would be willing to pay a large amount to see Taylor Swift in concert.

Such a demand also incentivized more people to try their hand at reselling for a large profit. Additionally, loyal fans who are adept at programming might have wanted to ensure they would get their hands on a ticket, and they knew how to use bots to do it.

While Ticketmaster was prepared for 15 million fans and the predictable number of bots, they didn’t have an architecture that could scale if online presence was double, triple, or in this case, nine times the anticipated amount. In other words, Ticketmaster prepared for big, but they couldn’t handle massive.

Verified Fan Program

Fans registered and verified as “non-bots” received an access code, which they would type on their website to enter the ticket queue. These access codes were a combination of letters and numbers. However, because of their random nature, these combinations would be easy for bots to guess and replicate, much like a password of 1234.

CAPTCHAs

Automation is not the only tech tool people can use to purchase tickets. While CAPTCHAs can prevent bots, they are not foolproof against AI, whose capabilities include image recognition. Most images on CAPTCHAS are from Google Earth, a platform easily accessible to AI programmers.

Several YouTube videos show how a bot that used AI clicked through a complicated CAPTCHA without issue. Most sophisticated websites no longer use CAPTCHAs for this reason.

Looking Forward: How You Can Use Tech to Reduce Bot Problems

Any website like Ticketmaster that offers access to high-demand activities like plays, sporting events or concerts must have a sophisticated and flexible response to an automation challenge like bots. Bots didn’t appear from thin air. They are intelligent tools used by intelligent people, and you can use them too.

Fortifying your website against unwanted bots is like playing chess. You must prepare for someone to uncover your strategy and beat it, and you must have a backup strategy on hand to counter your opponent’s next move. There is no easy, one-size-fits-all way to eradicate bots from your website because people can automate anything easy and repetitive.

Before jumping to preventative measures, you should first assess company policy on automation. If you decide you will not allow external bots – automation that isn’t from your organization – you can gather information about the types of users on your website by programming bots to analyze:

• Keystrokes: If keystrokes are impressively quick and cascade in a never-ending stream, they’re likely from a bot. You can program automation software to block users with this typing style. It might prevent the highly-skilled typist from getting tickets on occasion, but it can certainly block some unwanted traffic.
• IP addresses: If the same IP address appears dozens of times, it’s likely to be a bot, and if your company previously flagged the IP address, it is likely from a bot who was active at another popular online sale. There is no singular approach to prevent bots, but there are ways to limit certain suspicious behaviors. If you decide you will allow external bots, you can use automation to prioritize customer experience and fairness.

• Bot-specific platforms: A ticket-selling service could designate a certain number of tickets for bot purchase and leave the rest to human customers. Bots could have a separate platform to use, their method of accessing ticket sales different from humans to minimize the amount of crossover and to reduce website traffic. Alternatively, sellers could flag a certain percentage of tickets, perhaps the best ones, as not-for-resale. Because ticket brokers often use bots for the specific purpose of resale, fans who are likely to purchase non-refundable tickets could have exclusive access to better ones.

• Account creation algorithms: Organizations could program a procedure that tracks the time between account creation and ticket purchases to determine if a bot, a service or a person is attempting to use their platform. If they flag an account as a potential bot, they could ask the account holder to complete a human identity verification questionnaire. Meanwhile, the seller could either place those tickets back in inventory or on hold for a set amount of time.
• ID verification services: Most people are familiar with cookies – small files used to track your presence on a website – analyzing what you do and how long you stay. ID verifications work similarly, tracking your presence and actions on a website to determine if your behavior is more human or bot-like.
• Ticket origins: Taylor Swift isn’t the only artist who has had issues with Ticketmaster. Recently, thousands of Bad Bunny fans arrived at a concert venue in Mexico only to be turned away at the gate. Handheld scanners falsely read tickets as invalid, most of which attendees purchased on resale platforms. Additionally, several sets of tickets appeared to be sold twice, some on Ticketmaster and others on resale platforms.

Creating automation to determine where a customer purchased their ticket could limit tech malfunctions like these in the future.

Conclusion

Whether or how much you will allow external bots to have access to your website will guide the rest of your decision-making process, so you can automate tasks to make the user experience better while also developing strategies to help you problem solve when things don’t go as planned.