Advance robust AI and machine learning techniques and identify best practices for scaling AI

John P. Desmond, AI Trends Editor

Developing robust artificial intelligence and machine learning to reduce agency risk is a priority for the U.S. Department of Energy (DOE), and identifying best practices for implementing artificial intelligence at scale is a priority for the U.S. General Services Administration (GSA).

Here’s what participants learned during two sessions at World Government AI live and virtual event held in Alexandria, Virginia last week.

Pamela Isom, director of the Department of Energy’s Office of Artificial Intelligence and Technology, who spoke on “Advancing Robust Artificial Intelligence and Machine Learning Practices to Reduce Agency Risk,” has been involved in promoting the use of artificial intelligence across the agency for several years. With a focus on applied artificial intelligence and data science, she oversees risk mitigation policies and standards and focuses on applying artificial intelligence to save lives, combat fraud, and strengthen cybersecurity infrastructure.

She emphasized the need for the AI ​​project to become part of the strategic portfolio. “My office is dedicated to building a holistic view of artificial intelligence and mitigating risk by bringing us together to solve problems,” she said. These efforts are supported by DOE’s Office of Artificial Intelligence and Technology, which is working to transform DOE into the world’s leading artificial intelligence enterprise by accelerating AI research, development, delivery, and deployment.

“What I tell my organization to remember is that you may have tons and tons of data, but it may not be representative,” she said. Her team looks at examples from international partners, industry, academia and other agencies to identify results we can trust from systems that incorporate AI.

“We know that AI is disruptive in trying to do what humans do and do it better,” she said. “It goes beyond human capabilities; it goes beyond spreadsheet data; it can tell me what I’m going to do next before I even think about it. It’s so powerful,” she said.

As a result, careful attention must be paid to data sources. “AI is vital to the economy and our national security. We need accuracy; we need algorithms we can trust; we need accuracy. We don’t need bias,” Isom said, adding, “And don’t forget that you need to track the results of the models long after they’ve been deployed.”

GSA AI Work Executive Order Guide

Executive Order 14028, a detailed set of actions to ensure the cybersecurity of government agencies, issued in May of this year, and Executive Order 13960, promoting the use of trustworthy artificial intelligence in the federal government, issued in December 2020, provide valuable guidance in her work.

To help manage the risks associated with AI development and implementation, Isom has released an AI Risk Management Guide, which provides guidance on system functionality and risk mitigation techniques. It also has a filter of ethical and trustworthy principles that are considered across all AI lifecycle stages and risk types. In addition, the scenario is associated with the relevant presidential decrees.

And there are examples, for example: your results were obtained with an accuracy of 80%, but you wanted 90%. “Something is wrong here,” Isom said, adding, “The guide helps you look at these types of problems and what you can do to reduce the risk, as well as what factors you should consider when designing and implementing your project.”

The agency is currently internal to the Department of Energy but is considering next steps for an external version. “We will be sharing this with other federal agencies soon,” she said.

Outlines GSA best practices for scaling artificial intelligence projects

Anil Chaudhry, director of federal AI implementation at GSA’s Center of Excellence (CoE), who spoke about best practices for implementing AI at scale, has more than 20 years of experience in technology delivery, operations and program management in the defense, intelligence and national security sectors.

The Council of Europe’s mission is to accelerate the technological modernization of government, improve public experience and increase operational efficiency. “Our business model is to collaborate with industry experts to solve problems,” Chaudhry said, adding, “We are not in the business of recreating industry solutions or duplicating them.”

The Council of Europe provides guidance to partner agencies and works with them to implement artificial intelligence systems, as the federal government is actively involved in the development of artificial intelligence. “When it comes to AI, the government landscape is huge. Every federal agency has some sort of AI project going on right now,” he said, and the maturity of AI expertise varies widely across agencies.

Typical use cases he sees include focusing AI on improving speed and efficiency, cost savings and avoidance, improving response times, and improving quality and compliance. As a best practice, he recommended that agencies check your commercial experience with the large data sets they will encounter in government.

“We’re talking about petabytes and exabytes of structured and unstructured data here,” Chaudhry said. [Ed. Note: A petabyte is 1,000 terabytes.] “Also ask industry partners about their strategies and processes, how they conduct analysis of macro and micro trends, and what their experience is in deploying bots such as robotic process automation, and how they demonstrate resilience through data drift.”

He also asks potential industry partners describe the AI ​​talent on their team or what talents they can access. If a company doesn’t have enough AI talent, Chaudhry would ask, “If you buy something, how will you know you got what you wanted if you don’t have a way to evaluate it?”

He added: “Best practice for AI adoption is determining how you train your workforce to use AI tools, techniques and practices, and determining how you grow and mature your workforce. Access to talent will make or break AI projects, especially when it comes to scaling a pilot project to a fully deployed system.”

As another best practice, Chaudhry recommended looking into access to financial capital. “AI is an area where the flow of capital is very volatile. “You can’t predict or forecast that you’ll spend X amount of dollars this year to achieve what you want,” he said, because the AI ​​development team may have to explore a different hypothesis or clean up some data that may be opaque or potentially biased. “If you don’t have access to funding, that’s a risk that your project will fail,” he said.

Another best practice is access to logistics capitalfor example, the data that sensors collect for an AI IoT system. “AI requires a huge amount of reliable and timely data. Direct access to this data is critical,” Chaudhry said. He recommended entering into data sharing agreements with organizations related to the AI ​​system. “You may not need it right away, but having access to data so you can use it immediately and thinking through privacy issues before you need the data is good practice for scaling AI programs,” he said.

Latest Best Practice – Planning physical infrastructure, for example, data center space. “When you’re in a pilot, you need to know how much capacity you need to reserve in your data center and how many endpoints you need to manage” as the application scales, Chaudhry said, adding, “It all ties back to access to capital and all the other best practices.”

Find out more at World Government AI.