ML4Sci #36: Using AI to Disprove Graph Conjectures; Physical Systems for Backpropagation; AI4Science Data Brief

This is my thesis week 😪

Hi, I’m Charles Yang and I’m sharing (roughly) monthly issues about applications of artificial intelligence and machine learning to problems of interest for scientists and engineers.

If you enjoy reading ML4Sci, send us a ❤️. Or forward it to someone who you think might enjoy it!

As COVID-19 continues to spread, let’s all do our part to help protect those who are most vulnerable to this epidemic. Wash your hands frequently (maybe after reading this?), wear a mask, check in on someone (potentially virtually), and continue to practice social distancing.

---

Hi all,

Sorry it’s been a quiet month - I am squirreled away trying to finish my masters thesis in the next week, hence the shorter issue today as well. I’m hoping to ramp back up into ML4Sci newsletter with deeper dives on different topics over the summer!

First, some housekeeping. I published a data brief with some folks over at CSET@Georgetown on how AI is changing scientific innovation and some ways the US government can help accelerate this process. Writing this newsletter turned out to be a great practice for preparing this data brief!

Charles Yang @charlesxjyang

🎉Excited to release this data brief on AI4Science with the great folks over at @CSETGeorgetown! We provide qualitative and quantitative evidence of a growth in the use of AI in published scientific research + recs for USG on how to invest in this growing domain

Center for Security and Emerging Technology @CSETGeorgetown

👏New CSET Data Brief👏 Modern applications of #AI and #MachineLearning have the potential to change the practice of R&D in the U.S. over the next decade. But why and how? Matthew Daniels, @aut_toney, @flaggster73 and @charlesxjyang have answers: https://t.co/ToqTeJlkhn

And if you haven’t already, hop on over to join our ML4Sci Slack workspace and meet others interested in this space!

Department of Machine Learning

Facebook’s AI research team has made a big bet on self-supervised learning for computer vision and it is starting to pay off. In a recent ML4Sci issue, we covered SEER, a new self-supervised convolutional neural network that achieves new SOTA on ImageNet.

Now, Facebook has released self-supervised learning with transformers for learning image segmentation. The critical difference between CV and NLP was the ability of language models to learn from large amounts of unlabeled data - that gap is quickly closing with these new self-supervised techniques CV models.

Mike Schroepfer @schrep

Here’s our new computer vision system achieving state of the art results in image segmentation, without needing any labeled training data. This new model was trained on random, unlabeled data, but quickly achieved state-of-the-art results. It’s awesome.

Also, Facebook published a 12T parameter deep recommendation model. Lots of different tricks needed to efficiently train and run a model this big; the age of industrialized AI is here

😂After a new meme format out of XKCD went viral, dozens of variants have been appearing online. Here’s one that might be particularly relevant for readers of ML4Sci (also guilty of a few of these myself):

Tonio Buonassisi @toniobuonassisi

Inspired by @brunoehrler and @xkcd , an Applied ML version. I'm guilty of some of these myself. 😅

Near-Future Science

💼 Classified: Pfizer has an Applied Scientist - ML role in Cambridge

[QuantaMagazine] a nice overview of the main characters in AI for partial differential equations, including many articles/groups we’ve covered before!

Reinforcement Learning is starting to yield real returns to the scientific community. How does their use change the way we think about the scientific process and what scientific understanding means?

• [Arxiv] “Constructions in Combinatorics via NN”. Using RL agents to find counterexamples to several open conjectures in extremal combinatorics and graph theory!

• [Arxiv] “Experimental Deep Reinforcement Learning for Error-Robust Gateset Design on a Superconducting Quantum Computer”. Deep RL agents can also be used to improve error-robustness in quantum computing!

💻[Arxiv] “Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems”. Compute has traditionally meant electrons moving in silicon. New accelerators, like GPU’s or Cerebras’s massive new 2.6T transistor chip, are essentially smarter ways of moving electrons around. This paper shows that we can implement deep learning in *any* physical dynamical system.

The Science of Science

[NatureNews] Google scholar releases update to check if articles are conforming to open-access standards by funders.

🤦 [Verge] University of Minnesota was banned from editing the open-sourced linux kernel, after a research group there tried to insert cybersecurity flaws into the codebase to measure vulnerabilities in open-source software.

Policy and Regulation

📄[AIP] The House Science Committee has introduced a counterproposal to the Endless Frontier Act that came out of the Senate previously, that includes doubling the NSF’s budget.

Thanks for Reading!

I hope you’re as excited as I am about the future of machine learning for solving exciting problems in science. You can find the archive of all past issues here and click here to subscribe to the newsletter.

Have any questions, feedback, or suggestions for articles? Contact me at ml4science@gmail.com or on Twitter @charlesxjyang