In June, the U.S. Tennis Association (USTA) opted to hold the 2020 U.S. Open in New York (August 31-September 13) without fans due to COVID-19. That decision launched many changes, from electronic line calling to food-ordering apps for athletes. USTA has been working with IBM to integrate artificial intelligence for such uses as creating broadcast highlight reels based on crowd reaction. The June decision was “a pivotal moment,” said IBM’s sports and entertainment marketing program director Kristi Kolski. “A lot of the solutions that we had in the pipeline were no longer going to be viable.”

The Wall Street Journal reports that, “the USTA and IBM teams decided to focus on the fan experience, using AI-based techniques such as natural-language processing, or NLP, to create a number of new platforms for engaging audiences.” USTA managing director of ticketing and digital strategy Kirsten Corio explained that, “part of the reason for this approach … is that while the U.S. Open’s website and app serve around 10 million visitors every year, the tennis association expects viewership to be higher in 2020 owing to the pandemic.”

Among the AI efforts, the first is Open Questions, which uses NLP, to answer plain-text debate questions; “the platform searches more than 40 million pieces of unstructured data such as sports news and analysis articles … [and] extracts pertinent information, such as wins, losses and other records, and presents that as pros and cons for online debates.” Another project creates factsheets before matches.

IBM cloud platform chief technology officer Jason McGee reported that although “various U.S. Open tournament applications have been hosted in the cloud for years … pulling off an event without fans in the stands and limited staff on location required a full shift to a hybrid private-public cloud infrastructure.” On-site systems are now hosted in a private cloud while the fan experience is hosted in a public cloud.

VentureBeat reports that among IBM’s new technologies for the U.S. Open is AI Sounds, “which aims to recreate the ambient noise normally emanating from the stadium.” It leverages its AI Highlights platform to “digest video from last year’s U.S. Open,” ranks each clip for its “excitement level” and then classifies it for its crowd reactions score. It says that, “IBM used hundreds of hours of footage to extract crowd sounds, which it plans to make available to ESPN production teams that will serve it dynamically based on play.”

But the possibility exists that fans will find the resulting crowd reactions to be artificial, much like they have judged EA Sports’ Sound of the Stands platform, “which simulates crowd sounds using technology borrowed from the publishers’ FIFA game series.”

In addition to relying on Watson Discovery to facilitate the online debates, IBM is also using it to create Match Insights, “AI-powered” insights for the U.S. Open website and smartphone apps. According to IBM, its in-house digital design agency IBM iX developed AI Sounds and Watson Discovery features in 75 days.