The Facebook TransCoder Explained: Converting Coding Languages with AI

This new model converts code from a programming language to another without any supervision. It can take a Python function and translate it into a C++ function, and vice-versa, without any prior examples. It understands the syntax of each language and can thus generalize to any programming language!

Watch the video

This week my interest was directed towards TransCoder. A new model by the AI team at Facebook Research. Ask any questions or remarks you have in the comments, I will gladly answer everything!

Subscribe to not miss any AI news and terms clearly vulgarized! Share this to someone who needs to learn more about Artificial Intelligence! Spread knowledge, not germs!

The TransCoder Paper:

Video transcript

00:00this new model converts code from a00:02programming language to another without00:04any supervision00:05it can take a python function and00:07translate it into a c00:08function and vice versa without any00:10prior examples00:12it understands the syntax of each00:14language and can thus generalize00:16to any programming languages let's see00:18how they did that00:21[Music]00:25this is what's ai and i share artificial00:27intelligence news every week00:29if you are new to the channel and want00:31to stay up to date please consider00:32subscribing to not miss00:34any further news to understand this new00:36model by facebook00:38called the transcoder we first need to00:40introduce what it is00:41in short it's a trans compiler meaning00:44that it is a source to source translator00:46which converts a source code from a high00:48level programming language00:50such as c plus or python to another one00:54currently this type of translation00:55requires manual modifications00:58in order to work properly because of the01:00complexity of the task01:01it is hard to respect the target01:03language conventions01:04when changing from a programming01:06language a to a programming language b01:09plus since it is a complex task it01:12requires a lot of knowledge01:13in both target and source programming01:15languages to work01:16which means a lot of examples and01:19computation time01:20this is why facebook tried this new01:22approach using unsupervised machine01:24translation to train this fully01:26unsupervised01:27neural trans compiler the model was01:29trained on open source github projects01:32and mainly trained to translate01:33functions between three programming01:35languages01:36c plus plus java and python they01:38achieved that using a sequence to01:40sequence model with attention01:42composed of an encoder and a decoder01:44with a transformer architecture01:46trained in an unsupervised way01:48specifically on functions01:50at first they initializes the model with01:53cross-lingual mask language01:55model pre-training this is done by01:58randomly masking01:59some of the tokens and training the02:01transcoder to predict these mask tokens02:04based on the context allowing the model02:06to create high quality02:08sequence representations regardless of02:10the programming language02:12once the pre-training is done the02:13decoder is then trained02:15using the pre-trained transcoder as02:17input to always generate02:19valid code sequences even when fed with02:21noisy data02:22increasing the encoder robustness to02:24input noise02:26in order to translate functions from one02:28language to another02:29we need to add this last part the back02:31translation02:33which is composed of two models as you02:35can see in this image02:36the target to source model which is used02:39to translate the target code into the02:41source code02:42producing a noisy source sequence then02:45this sequence is used in the source to02:47target model to reconstruct the target02:49sequence02:50from this noisy input if this doesn't02:53tell you anything02:54i invite you to check out the videos i02:56made on attention02:57and transformers which are linked in the02:59description below03:01as the results show there are still a03:03lot of progress to be done in the field03:05but the best translation shows over 9003:08success03:08which is a huge jump in accuracy here03:10are some examples of functions03:12translated using the transcoder03:15i invite you to pause the video and take03:17a deeper look at these examples03:35programmers will be able to easily03:37improve and build03:38robust cross-platform software with03:41these improvements in translations03:43with little or no modifications to make03:45to their code03:46using a single code base the code and03:49pre-training models will be publicly03:51available soon as they said in the paper03:54of course this was just a simple03:56overview of this unsupervised code03:58translator03:58i strongly recommend to read the paper04:00link to the description to learn more04:02about it04:02and try out the code whenever it's04:04available leave a like if you went this04:06far in the video04:08and since there are over 90 of you guys04:10watching that are not subscribed yet04:12please consider subscribing to the04:14channel to not miss any further news04:16clearly explained if you want to support04:18the channel i now have a patreon04:20linked in the description where you can