If you want to figure out how to get the most out of a brand new supercomputer, try crowdsourcing.

That’s the idea behind the Intel® Xeon Phi™ User's Group (IXPUG), the user group for the new Intel Xeon Phi “manycore” processor that is the heart of the new Theta machine recently installed at the U.S. Department of Energy's Argonne National Laboratory as a stepping-stone to its next-generation supercomputer, Aurora.

In September, Argonne hosted over 130 IXPUG participants from around the world for four days of tutorials, workshops and talks aimed at illuminating the still relatively unplumbed world of the Xeon Phi processor. Users, programmers and systems administrators collaborated to share experiences, trade tips and help one another optimize code for applications ranging from simulating brain tissue to modeling the evolution of the cosmos.

“It really helps to get a group of people together who are all working in the same space to share what the reality is,” said David Martin, a member of the IXPUG steering committee and manager of industry partnerships and outreach at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility.

IXPUG got its start four years ago when the first Xeon Phi came out. “Intel wanted to support the users, and users wanted to talk about the differences between these processors and regular processors and how people were starting to use them,” said Melyssa Fratkin, the group’s secretary and director of industrial programs at the Texas Advanced Computing Center (TACC), which has a first-generation Phi machine and hosted the first IXPUG meeting.

The new processor, named “Knight’s Landing,” is the second-generation Xeon Phi chip. (Generation One was “Knight’s Corner;” Gen Three, which will run the Aurora system scheduled for arrival at Argonne in the 2018 timeframe, is called “Knight’s Hill.”) 

A novel feature of Knight’s Landing is that it allows memory to be used in several different modes. Each processor has 64 cores –  compared to 16 on the chip in the ALCF’s Mira supercomputer – which delivers computational power in a very energy-efficient way, making the chips ideal for large-scale scientific computing.

All of this means that large-scale scientific codes that have sometimes taken decades to write must be adapted for the new architecture and its new capabilities. Early users at Argonne, Lawrence Berkeley National Laboratory, Los Alamos National Laboratory and the TACC are all running such programs on the Phi architecture, working to figure out what works best.

It is uncharted territory. “Intel designed the Xeon Phi to be flexible, but they didn’t do a whole lot to recommend the best way to use it,” said Martin. “If your system is not working at all or one of the memory modes has failed, you can call Intel and they’ll help you. But if you’ve got a cosmology code that‘s not working very well, you rely on the community.”

The community was present in force at the September meeting.

“It’s a very eclectic and very international group,” said Fratkin.

Some participants had little experience with the Xeon Phi system and were there for the tutorials that would teach them how to port their code to the Phi. Others with more experience came to share what they’d learned. There were talks about programming, talks about how to use the various memory modes and people talking about what they did with specific applications. There were opportunities for hands-on experience as well; participants ran debuggers or logged in and tried programming exercises.

But according to Martin, the unscheduled time was at least as important as the program:  “We set it up for a lot of hallway discussions; people sharing experiences over lunch or during a break. To talk to someone who’d already solved a programming problem with the Intel Xeon Phi who could say, ‘I tried these seven things and here’s what worked for me’ – I think that’s what’s valuable.”

Valuable for Argonne, too, as it gets to know the new Theta system. Argonne is a very early adopter of new supercomputing technology – the Theta system is only the second or third machine of its kind ever built.

“We always joke that we buy a single-digit serial number machine,” Martin said. “We have machines before they’re really fully baked and we rely on the community to figure out how to make them work.”

And, said Mike Papka, director of the ALCF, “It’s really important that everybody is not solving the same problem over and over. How do we share knowledge, best practices, lessons learned? IXPUG brings that to the table. And the nice thing about it is it’s done in collaboration with Intel, so they’re sharing what they’ve learned with the community of users.”

The user community will likely expand considerably as the new Phi architecture gives more people access to supercomputing power, said Papka. The user community for supercomputers like Mira is limited to those who could afford the couple of million dollars it took to get started with the machine.

“With the Xeon Phi architecture, a university professor can buy a desk-sized machine with a couple of Xeon Phi nodes ,” Papka said. “It allows them a place to get started, and then they can move from their desktop to a campus-wide infrastructure and on to something like Argonne’s Leadership Computing Facility.”

IXPUG, Papka said, spans the whole space of users. “I think that’s really where the strength lies, in building this user community,” he said. “The future of all computing is going to be parallel processing. Having a user community that embraces that, that understands that, that supports each other, is hugely important.”

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science.

The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.