Anton Popov, Artificial Intelligence /Deep Learning Technical Lead at Ciklum
The 2018 International Conference on Machine Learning (ICML 2018) took place in the middle of summer in Stockholm, Sweden. With a wide variety of incredible, ground-breaking sessions, attendees likely had a difficult time deciding between them. One of these options included an excellent panel discussion called “Machine Learning That We Need.”
This discussion was part of the Fairness, Interpretability and Explainability workshop. In this session, panelists discussed issues and information gaps that are currently creating a buzz in the Machine Learning community.
The panelists discussed some of the fundamental problems that have become increasingly evident as Machine Learning has matured in terms of both technological advancement and increased public awareness. The main question the panel posed was,
Additionally, the panelists touched on the best way to consider the ethical and legal ramifications of the decisions made by AI. They also asserted that society must work collectively to develop approaches to ensure that Machine Learning models align with the public good.
Designing Solutions for the Greater Good
The panelists posed the idea that to ensure fairness and alignment with the greater good, domain experts from fields outside of Machine Learning should be deeply embedded in Machine Learning projects. This offers the opportunity for projects to be created in order to promote the greater health and well-being of humanity as opposed to simply creating an advantage for a certain subset of well-informed individuals or groups.
For example, Machine Learning can be used to create a chess master in a short period of time — see Google’s experiments with DeepMind — but does this really accomplish a significant impact in contrast to applying Machine Learning to health or medical settings? The panelists considered how to apply Machine Learning and AI in order to solve real-world problems.
One of the issues that currently impacts Machine Learning, the panelists said, is the fact that there is currently a substantial gap between research and real-world application in the Machine Learning domain. While research tends to focus on the more technical aspects — like architecture, optimization, interpretability, and scalability — it does not always translate to achieving positive outcomes.
The Machine Learning community has to consider how to facilitate important conversations between technical architects and those in other fields in the social sciences, humanities and other disciplines that can provide important data to inform Machine Learning projects. This is an important step in ensuring that growth in Machine Learning coincides with positive social outcomes.
For example, NVIDIA demonstrated how AI can be used to fix photos:
This approach could one day help people fix their favorite holidays snaps and have even bigger implications: in healthcare the technique could be applied to enhance MRI images.
Additionally, the panelists stated that the Machine Learning specialists themselves have to be more involved in the collection of experimental data as they design programs in order to understand the roots of the problems they’re trying to solve. By being part of the process early on, solutions can be designed with the end in mind.
There are, of course, some issues that outside the realm of knowledge for Machine Learning specialists, such as privacy, consent, ethical and legal challenges. By teaming up with subject matter experts in those areas, however, the specialists designing the solution can gain a better understanding of how to handle these concerns while creating exciting new products and services.
Broadening Understanding of Machine Learning and AI Concepts
The panelists also discussed the need for enabling wider distribution of information about Machine Learning and AI concepts to the public. There is currently a great deal of hype around Machine Learning, and also some misconceptions about what Machine Learning is (and isn’t) as a result. Many people think of AI as having the ability to learn in an out-of-control manner by acting autonomously and being exponentially scalable.
There is a need in philosophical and terminological disputes on the essence and role of “intelligent” machines in our life, but we have to separate this from operational meaning of the “artificial intelligence”. Actually, AI is just a name of the set of technologies which can automate tasks previously feasible only to human.
The panelists emphasized the importance of the Machine Learning community being open and honest about both the capabilities and limitations of AI and Machine Learning in order to curb fantasy. One way to demystify Machine Learning and the technology “under the hood” could be to utilize designers to make AI functionality more available and usable by the general public.
The panel also discussed the issue of fragmentary or incomplete knowledge of Machine Learning concepts by those engaging in projects. They discussed the problem of mistakes being made due to misunderstanding the capabilities and functionality behind common Machine Learning solutions. The results of many previous studies and tests have been impossible to recreate, and many conclusions that were previously thought to be true have been called into question as a result. This makes it difficult to determine what has actually been accomplished according to true Machine Learning principles.
Moving forward, universities and online learning platforms will need to provide structured, coordinated education so that Machine Learning is taught the right way and not compromised by incorrect usage or fragmentary knowledge.
Get Your Whitepaper: “Artificial Intelligence, Machine Learning and Deep Learning: the Difference Explained”
Theory vs. Practice
The panel also examined the gap between theoretical findings and practical applications.
The first point they made was that model deployment is often very difficult in actuality. There are many tools now available to assist with the process, but previously, Machine Learning engineers were responsible for designing appropriate solutions. Often these were case-specific or designed according to the need they had at the time. This doesn’t necessarily mean that that model would work in other instances. Additionally, this can lead to a loss of design function, which cannot be seen during development but is realized after deployment.
The second point the panel made on this topic was about demonstrating the usefulness of Machine Learning as a discipline to various industries. Many are still skeptical of the efficacy of Machine Learning and AI as it relates to real-life situations. It is up to the Machine Learning community to find a way to effectively convey who it can be used in concrete ways. It can be difficult to bridge the gap from theoretical constructs like training, loss, inference and learning to quantifiable and observable behaviors.
The third point made by the panel is that Machine Learning must be open to counterexamples and criticism. In order to better define what Machine Learning is capable of, the community must also be able to define what it doesn’t do. This includes defining parameters of usage and understanding how to outline best practices, while use the enthusiasm and momentum of AI to produce exciting new solutions.
Black Box vs. Transparent Models
This need for transparency in Machine Learning performance is ideal; however, it has been shown that many black-box models of testing often demonstrate better performance. The need for interpretability is often important and valuable, but not always necessary.
The panel discussed how both black-box and transparent models can be useful and are not mutually exclusive. They offered that, in many cases, you are able to start with the interpretable model as a first choice and then move on to a black-box model for more complex use cases or to further test capabilities. There are even examples of interpretable analysis of black-box models.
Making Machine Learning Tools More Accessible
Another way of promoting good practices is by releasing information on code and data sets. Machine Learning practitioners should have access to examples and solutions designed for situations that are similar to what they’re designing a solution for.
The panel noted that this is often difficult to achieve, however, due to the fact that source code and data is often private and sensitive. Whenever possible, though, data sheets could be prepared to support reproducibility by removing names and other private information. Some organizations and universities do share data sets, like UCI. In general, sharing best practices along with work samples is a very effective way to promote Machine Learning and AI best practices for specific design models.
The engaging and high-energy session concluded with the message that Machine Learning will soon be a major part of everyday life for most people in one form or another. The possibilities and potential for growth are essentially limitless, and this is a very unique and exciting time for Machine Learning and AI. As this growth occurs, however, Machine Learning professionals and practitioners in other disciplines need to find a way to ensure that the right kind of impact is being made. If the thrill of Machine Learning technology can be harnessed in order to creatively solve real-life issues, its true potential as a change agent will be realized.
Need to build an AI-powered system? Check how Ciklum R&D team built AI model to detect fruits in nearly human-level performance for SeeTree . Drop us a few lines into the website contact form and we’ll reach you back in 24h.