A Paradox Of Neural Encoders And Decoders Or Why Don`t We Talk
-
Rating
-
Date
September 2018 -
Size
145.8KB -
Views
4,625 -
Categories
Transcript
"!#$ % & ' ()"!# *,+.-/' 10&3254/627% 98;: ;< =?>A@CBEDGFH6IKJCLMDGNPORQCS$BT>UJ<;BT>UVE=GOWQC>UJC@ XY>AJCDGZ$[VEB\DPNGO?]E^ _`;acbYdPegf.hagiMf;jPkml6jRh$bKnYf.acepoUqcrsaciYq3adPiYt uwvsa3q3fxe.rsq?dPvyuwi{zGr|i{a3age.r|i{z } oUq.~{jUjGvwjPkyy.Uq.~{jGvsjWzW iM ;rsWac`egxrsfW WjP$km n{nY.a3fxacegiYdGxv|vsrsdPd iYt MMPUYU YUKGwUPYWU M¡W¢U£UURw¤¦¥U§y¤xW¨W¥©¤.¥ a tMa3WagvsjPb³diYa3´kEegdWhµa3´6jPe.¶·kTjPeµ.fxn{tMWr|iYz¸fx~Ya&¹MrsdW.a?&fx~YdWf geªa3qc«wnM¬3e.3eg®Gac¯MiM°UfWiY±acnM² egdPvMi{a3f´6jPe.¶Mw¹Me.r|i{zf.jvsdPiYzPnYdWzWabKegjUq?a3.xr|i{z f.dWx¶M3ºR».a3hdGiMfxrsq q3jGi{q3agb{f$egagbKega3.agiMf.a3t¸¹Md*b¼jPr|iMfr|i,u©nYqcv|rstUrsdPi½xb{dRq3a&rsfxegdGi{xvsdRf.a3t¸r|iMf.jd .Uh¹CjPv9.a3¾WnYaciYq3a&¹M dGi,aci{q?jUtKaceµi{a3f´6jPe.¶Cºy¿6~Mrs.a3¾WnYaci{q?a rsfx~{agi¸kÀa3t f.j dtMa3q3jUtMacei{a3f´6jPe.¶·´6~Krsq.~³dRf.f.a3h$bYf.$f.jfxegdPiYxvsdWf.arsf$¹YdWq.¶»f.jÁfx~YajPe.rszGr|i{dGv q3jGi{q3agb{f3º ² a&x~YjG´~{jG´Âd*bYdPr|ejPkega3qgnKe.egaciMf i{a3f´6jPe.¶MµdWq3fxr|iYz dW aciYq3jUtMace dPiYtÃtMa3q3jUtMaceÄq3dPiÅtKa3RacvsjPbfx~Yacr|eÄjP´6iÅ.UhƹCjGv|rsqÁvsdPiYzPnYdWzRaÁfx~YdWfrs.ace.rsdPv|vs fxegdPiY.h$rsf.f.a3t»¹Ca3f´6a3aci,fx~Ya3hacrsfx~YacekTjGeg´6dPegtMjPeµ¹{dRqǶM´6dGegtK3ºy¿6~Ya aciYq3jUtMace dPiYttMa3q3jUtMacew¹Ke.r|iYztUr|ÈCacegaciMfyq3jPiY.fxegdPr|iMf.wf.jfx~Ya6f.dWx¶CÉGdGi{tfx~Ya3.a6a?dPe.vs;ega3xnMvsf. r|iYtArsq3dRf.a$fx~YdWfmfx~{a9q3jPiMÊKrsq?fxr|i{ziYdWfxnMegajPkfx~{a?.aq3jPiY.fxegdPr|iMf.hdG$¹CaegacÊYa3q3f.a3t r|i,fx~{avsdGi{zGn{dRzWafx~YdWf$nMvsfxrshdWf.agvsÃa3hagegzWa33ÉbMegjGWrstAr|iYz rsh$bCjGegf.dPiMfqcv|nYa3f.j fx~Ya.fxe.n{q3fxnMegajPkw~UnYhdGivsdPiYzPnYdWzWa?3º Ë ÌKÍÎ{ÏÐÑ$Ò ÓÎ{ÔgÐÍ H ÕCD NcZgÖC@×FIUØÙ>AÖCZgIKÚ·>AZ?> >AJC@ZcÕCDBT>UJCÛKÖ>UÛKDGNZgÕ×DGFÜR>AJÝC=cI{ÜGDGNgN$Õ>UN>ÕCVENcZgIM=cF@>UZcVEJCÛ Þ >UÜ3LZcIH6ÖC=cV\J×Ûß|àWám>UJ×@ Þ DGFMIKJC@yâCã6J{Z.äVEJCDG@ÁäV\ZcÕZcÕCVENNgZgIK=gFVEN;ZgÕCDNgZcÖC@CFÁIUØmJ>UZgÖ×=?>UB BT>UJCÛKÖ>UÛKDGNy>UJC@&IUØZcÕCD9Õ{ÖCÚ >UJÚVEJC@åâ3HÕCD9VENcNgÖCD;VENyDGNcNgDGJ{ZgVT>AB\BEFIMJ×D9IAؼÜPIMJCNcZg=?>AVEJYZcNRâUHÕ×D Ý>U=cZgVEÜGÖCBT>A=yÜGIKJCNgZc=?>UVEJ{ZgNIMJ>UJ>AÖCZgIKÚ·>UZcIMJyQPNgÖCÜ3Õ>UN6ZgVEÚÁD9>UJC@ NgÝ>UÜGDMQAÝCBT>UÜGD Þ IKÖCJC@CNIMJ ZgÕCD&Z.FYÝDGN$IUØZ?>ANgL{N VEZÜR>AJ»ÝDP=.ØIK=gÚæNgÖ×Ü?ÕÃ>UN ZcÕCDZ.F{ÝDGN IAØÙBT>AJCÛMÖ>AÛMDGNVEZÜR>AJ»ÝC=cIYÜPDGNgNWâ çwVELMDPäV\NcDMQGVEZwVEN Þ DGBEVEDGèMDP@ZgÕ>AZyZcÕCDÜGIKJCNgZc=?>UVEJ{ZgN6IAؼZcÕCDÕ{ÖCÚ >UJ ÚÁVEJC@µ>A=gD=cD3éDGÜGZcDG@VEJZgÕ×D BT>UJCÛKÖ>UÛKDGN9ä;DÖCNgDKQCNgIZcÕ>UZ Þ FÁDGêC>UÚVEJCVEJCÛ&ZgÕCD$ØDR>AZgÖC=cDGN IAØBÀ>AJCÛMÖ>UÛMD$ä9DÚ >WF Þ DGZgZcDG= ÖCJC@CDP=gNgZ3>UJC@¸ZgÕCDÝC=cV\J×ÜGVEÝCBEDGNZgÕ>AZÛMÖCVE@CDÕ{ÖCÚ >UJBT>UJCÛKÖ>UÛKDµ>AJC@ ZgÕCIKÖCÛMÕ{Z;ÝC=gI{ÜGDGNcNgDGNWâ ëyDP=gÕ>AÝCNZgÕCD Þ DGNgZL{JCIUäJÃä9IK=gL =gDPBÀ>AZgVEJCÛ >UÖCZcIMÚ·>AZ?> >UJC@ÃBT>UJCÛKÖ>UÛKDGNRQäÕCVEÜ3Õ,>AB\NcI NgDGDPÚN Õ×V\ÛKÕCBEF =gDGBEDGèM>AJYZZcIJ>AZgÖC=3>UBBT>UJ×ÛMÖ>AÛMDGNWQCVENì;ÕCIMÚÁNgL{Fwí NÕ×V\DP=?>U=cÜ3ÕYFîß\ïágâì;ÕCIKÚð NgL{Fwí NÕCVEDG=?>A=gÜ3Õ{FVENp>$Ø.>UÚVEBEFIUØyBT>UJ×ÛMÖ>AÛMD;ÜGBT>UNgNcDGNZgÕ>UZ;ÜR>UJ Þ D$=gDGÜGIKÛMJCVENcDG@ Þ F>ÜGIK=g=cD3ð NgÝIMJC@×V\J×Û;Øñ>UÚÁV\BEF;IUØ>UÖ×ZgIMÚ >UZ3>ÜGBT>ANgNgDPNRâU[VEZcÕµ@×VsòwDG=cDGJ{Zm=cDGNgZc=gVEÜGZcV\IKJCN6IMJZgÕ×Dp>AÖCZgIKÚ·>UZ3>CQ @CVóòôDP=gDGJ{ZBT>AJCÛMÖ>AÛMDÁÜGBT>UNgNcDGNÚ >WF Þ D»ÝC=cI{ÜGDGNgNcDG@yâõ$IUä9DPèMDG=WQZgÕ×V\NÕ×V\DP=?>U=cÜ3ÕYFîäp>AN@CD3ð NgVEÛMJ×DG@äVEZgÕîNgF{Ú Þ IMBEVEÜNcFYNcZgDGÚÁNVEJîÚVEJC@yâwö.Z Õ>AN Þ DGDPJ½NcÖCÛMÛKDGNgZcDG@ZcÕ>UZ @CF{J>UÚÁVEÜR>UB ÷ ²xnMbKabCfxjG~{egdGf.iKa3¶Ät¸¿w¹MjP·iMdPi³wvs9dWCf.a ùdPiYf.jÁt ú6uwv|¿rsø3ÉwdGd¹Ca3fx~, ùoWw¶UvsjWdP.e9f.tMkTjPjUeq3f.~YjGacegv|dPbKv9k\nKû{vmacv|tAvsjGrs.´qgn{x~K.xr|b½rsjGi{f.3jº;¿6úÂ~{adPegiYa3tÃ.a3dGdGegiÃqÇ~Ä9´6üdWl zPegdGiMff.j$ý ² º NgF{NgZcDGÚNWQUVEJCÜGBEÖC@CVEJCÛ$Ú·>AJYFÜGIKJCJCDGÜPZgVEIMJCVENcZÚI{@CDGBENWQRÚ >WF Þ =gVEJCÛ@CVóòwDG=gDGJ{Z Þ VT>UNcDGNZgIBT>UJ¼ð ÛMÖ>AÛMDÝC=cIYÜPDGNgNcV\J×Û·Z3>UNcLYN$=gDGBT>AZgVEèMDZgIÁZgÕCDPV\=$NgF{Ú Þ IMBEVEÜÜGIMÖCJ{ZcDG=gÝ>A=gZcNÄß|þáÇQ{J×DGÜGDGNcNgVEZ?>AZgVEJCÛ >=cD3ð.DGèM>UBEÖ>AZgVEIMJÁIUØmZcÕCD>AÖCZgIKÚ·>UZ3>MÿABÀ>AJCÛMÖ>UÛUD=gDPBÀ>AZgVEIMJ×NgÕCVEÝyâ S Nä;DGBEB >UNÝC=cIYÜPDGNgNcV\J×ÛÜGIKJCNgZc=?>UVEJ{ZgNWQÜGIKJCJCDGÜPZgVEIMJCVENcZÚI{@CDGBEN>UBENgIÃÕ>WèMD ÜGIMJ×NgZg=3>UVEJ{ZgNWâHÕ>UZ VENWQZcÕCDGF,>A=gDBEVEÚVEZcDG@JCIKZ IKJCBEF·VEJ»äÕ>UZ ZcÕCDGF»ÜW>UJ»=cDGÝC=gDPNgDGJ{ZRQ Þ ÖCZ VEJäÕ>AZZgÕ×DGFÜR>UJBEDR>U=cJyâHÕ×D»@CVENgZcV\J×ÜGZgVEIMJ Þ DGZ.ä;DGDGJB\DW>U=gJ×V\J×Û>UJC@=gDPÝC=gDGNcDGJ{Z?>AZgVEIMJ VENV\ÚÁÝIK=gZ3>UJ{ZäÕ×DGJä;D»ÜGIMJ×NgVE@CDG= ÕCIUäÕ{ÖCÚ·>AJBT>UJCÛKÖ>UÛKDGNÕ>WèKD»@CDGèKDGBEIMÝDG@yâ ×IM= > J>UZcÖC=?>ABCBT>UJCÛKÖ>UÛKD9ZcI Þ D$èYVT> Þ BEDMQWV\ZÚÖCNgZåJCIMZIKJCBEF Þ D=gDGÝC=cDGNgDPJYZ3> Þ BED Þ FVEZcNmÖ×NgDG=cNRQ Þ ÖCZ >UBENgI BEDR>U=cJ> Þ B\D Þ FÁNgÖ Þ Ng D {ÖCDGJ{ZÛMDPJCDG=?>AZgVEIMJ×Nß Wágâ{HÕCDBEDR>A=gJCVEJCÛ>AJC@»=cDGÝC=gDPNgDGJ{Z?>AZgVEIMJ>UB ÜGIMJ×NgZg=3>UVEJ{ZgN$IUØmZcÕCDÕ{ÖCÚ >UJ Þ =?>UVEJ @CVEÜGZ3>UZgD&ZgÕCDNcDGZ IAØBÀ>AJCÛMÖ>UÛMDPNÕYÖ×Ú·>UJ×Np>A=gD> Þ BEDZcI ÖCJC@CDP=gNgZ3>UJC@Ã>UJC@ÁB\DW>U=gJåQ>UJ×@·ÜPIMJCNc D {ÖCDGJ{ZgBEFZcÕCDBT>UJ×ÛMÖ>AÛMDGN;ZgÕ>AZÕ>WèMDDGÚÁDG=gÛKDG@yâ DGÜGÖ×=g=gDPJYZ&JCDGÖC=3>UBJCDPZ.ä9IK=gL{N N Õ>WèMD NgÕCIUäJ»NcVEÛMJCV ÜR>AJ{Z ÝC=cIMÚVENcD>UNÜGIKÚð ÝCÖCZ3>UZgVEIKJ>UBÚI{@CDGBEN;IUØèM>U=cVEIMÖCN>UNcÝDGÜPZgNIUØZgÕ×DµÕ{ÖCÚ >UJÁBÀ>AJCÛMÖ>UÛMD$ÝC=gI{ÜGDPNgNgVEJCÛÁNgF{NgZcDGÚ,â ë6>U=gZ6IAØZgÕCDPV\=6Ú > 3IM=y>AÝCÝDR>UBCVEN6ZgÕ×Dp> Þ VEB\VEZ.F$ZgIVEJCÜGIM=cÝIK=?>AZgD;NgF{J{Z?>Uê >UJC@ NgDGÚ >UJ{ZgVEÜGNåVEJYZcI >ÁNgVEJCÛMBEDDGJ×ÜGIMÚÁÝ>UNcNgVEJCÛ ÚI{@CDGBß WágâwHÕ×DGF,Õ>WèMD>UBENcI·@×DGÚIKJCNgZc=?>UZcDG@³ÜGIKÚÝDGZgDGJ×ÜGDVEJ BEDR>U=cJCVEJCÛ>&äV\@×Dµ=3>UJCÛKD IAØmÛM=3>UÚÁÚ·>UZcVEÜR>UB{NgZc=gÖCÜPZgÖC=cDGNß ágQM>UJC@ IAØZgDGJÁ=gD3éDGÜGZ =cDR>UBóð.ä9IK=gBE@ @>UZ3>IKJ J>UZcÖC=?>ABwBÀ>AJCÛMÖ>UÛMD$Z?>ANgL{Nß !CQG#ï "WágQC>UJC@ÁBT>UJCÛKÖ>UÛKD Ü3Õ>UJ×ÛMDß $Wágâ{S BEZgÕ×IMÖCÛKÕZgÕ×DGNgD =gDGNcÖCBEZgNÕ>WèKD Þ DPDGJÅ>òôDPÜGZgDG@ Þ F VEZRQ×ZgÕCD VENgNgÖ×DIAØ9ÜGIMJ×NgZg=3>UVEJ{ZgNIUØ %& NÕ>AN JCIKZ Þ DGDGJ DGê{ÝCBEVEÜGVEZgBEF DGêC>UÚVEJCDP@yâö.ZµNcDGDGÚÁNVEÚÝIM=gZ3>UJ{ZZgÕ×DGJyQåZgI VEJYèKDGNgZcVEÛY>UZcDZcÕCDÜGIKJCNgZc=?>UVEJ{ZgN&IUØ =gDGÜPÖC=g=cDGJ{ZJCDGZ.ä;IM=cLYN>AJC@ZgÕ×D9ä>WF ZcÕ>UZ6ZgÕ×DGFV\J¼éÖCDGJCÜPDZgÕCD;ÝC=gIKÝDP=gZgVEDGN>AJC@DPÚDG=cÛMDGJCÜPD IUØ6BÀ>AJCÛMÖ>UÛMDKâ HÕCVENmÝ>UÝDG=VEN6ÚIMZcVEèM>UZgDP@ Þ FZgÕCDI Þ NgDP=gèM>UZcVEIMJ ZgÕ>AZmÜPIMÚÚÖ×JCVEÜR>UZcV\IKJ VENDGNcNgDGJ{ZcVÀ>ABEB\F >;NgÕ>A=gDG@&Z?>UNcL Þ DGZ.ä;DGDGJ NgDPJC@CDG=>UJ×@=gDGÜPDGVEèMDG=WQRVEJäÕCVEÜ3ÕZgÕ×DL{V\J×@µIAØCBÀ>AJCÛMÖ>UÛMDåØñ>WèKIMÖC=cDG@ Þ FZgÕCD$NgDGJ×@CDG=;Ú·>WFJCIMZ Þ DÜGIKJ{èMDGJCVEDGJ{ZØIM=6ZcÕCD =cDGÜGDGVEèMDP=p>AJC@è{V\ÜPD3ðñèKDG=gN3>CâKHÕ>UZV\NWQUZgÕ×D ÜGIMJ×NgZg=3>UVEJ{ZgNIAØÙZcÕCDNgDGJ×@CDG=>UJC@ =cDGÜGDGVEèMDP= Ú·>WF Þ D@CVóòôDP=gDGJ{ZRâHÕCDBT>UJ×ÛMÖ>AÛMD ZcÕ>UZÖ×B\ZcVóð Ú·>AZgDGBEF DGÚDG=cÛMDGNÚ >WFÄ>A=gVENgD>AN > ÜGIMÚÁÝC=gIKÚVENgD Þ DPZ.ä9DGDPJ,ZcÕCDGNgD&ÜGIMÚÁÝDGZcVEJCÛVEJ{ZgDP=gDGNcZgNRâ [³D ÜGIKJCNgVE@CDG= >&NgVEÚÁÝCBEDBÀ>AJCÛMÖ>UÛMDZ3>UNcLVEJäÕCVEÜ3Õ·Z.ä;I &$N9Zc=gFÁZgI ÜGIMÚÁÚÖCJCVEÜR>UZcD > NgDGÚ >UJ{ZgVE(Ü 'gÜGIKJCÜGDGÝC*Z ) =gDGÝC=cDGNgDPJYZcDG@ Þ F·> ÝIMVEJ{ZVEJ > NcÖ Þ NcDGZR,Q +.-ö 0/·IAØ9ã6ÖCÜGBEVE@CDR>AJ NgÝ>AÜGD»2â 1$JCDJCDGZ.ä;IM=cL»NgDGJ×@CNÄ> 34 55 6 VEJÃZgÕCD&ØIM=cÚ IAØp>ÁNg D {ÖCDGJ×ÜGDIAØ Þ VEZcNRQwäÕCVEÜ3Õ ZgÕCD$IMZcÕCDG=JCDGZ.ä;IM=cL=gDGÜPDGVEèMDGN>UJC@Á@CDGÜGI{@CDGN Þ >UÜ3LVEJ{ZgI>&ÝIMVEJ{Z9VEJZgÕ×DµN3>UÚÁD ã6ÖCÜGBEVE@CDR>AJ NgÝ>AÜG7D åVEÛâM#ï GâPö.JZcÕCVENyÝ>AÝDG=6ä;DÜGIKJCNgVE@CDG=6IKJCBEF ZgÕ×D9ÜR>ANgD;äÕCDG=c&D +VENyZgÕ×DÖCJCVEZyVEJ{ZgDP=gèM>UB ß "98Uï3:á -ö &â [ÕCVEBED9ZcÕCD Z3>UNgL VENNgÖ×ÝDG;= ÜGVT>UBEBEF NgVEÚÝCBEVENgZcVEÜMQWV\Z6Õ>AN9>J{ÖCÚ Þ DG=6IUØyVEJ{ZgDG=cDGNgZcVEJCÛÝC=cIMÝ{ð DG=gZcVEDGNRâ{HÕC(D 'gÜGIKJCÜGDGÝ×UZ VEZäp>AN {Ö>AB\VEZ3>UZgVEèKDGBEF,NcV\ÚÁVEBÀ>A=äV\ZcÕ=cDGNgÝDGÜGZ ZgI³ZcÕCD BEDR>U=cJCVEJCÛ Z?>ANgLIAÁØ N{DPÜGZR2â !Câ !×Q Þ ÖCZ;Øñ>ANgZgDP=Râõ DGJ×ÜGDMQw> ÕCVEBEBsð.ÜGBEVEÚ Þ DG=ä>UNÖ×NgDG@ ØxIM=ZgÕ×DDGJCÜGI{@CDP= >AJC@
UNcVEÜÄ>AB\ÛKIM=cV\ZcÕCÚ/ØIK=ZgÕCD ÜGIUð.DGèKIMBEÖCZgVEIKJ·IAØ9ZgÕ×DNgF{NgZcDGÚ VEN9@CDPNgÜG=cV Þ DG@ÃVEÃJ åVEÛâ Câ sWº låega3dRf.adq.~{dRh$bKrsjGi agi{q3jUtMacepdPiYt tMa3q3jUtMaceGº Aº låega3dRf.adhn{f.dGiMfagi{q3jUtMace9¹M&bCacegfxnMe.¹Mr|i{zfx~{a´6agrszP~Mf.;jGkwfx~YaqÇ~YdWh bKrsjGiº ÚK&º hEkfx~YaaciYq3jUtAr|iYz qcega?dWf.a3tÁ¹Mfx~YahÆnYf.dPiMfnY.a3$dzGega3dWf.ageGdPe.rsa?fxjPk.fxe.r|iYzWfx~YdPi fx~{a q.~YdWh$bMrsjPiÉ.agvsa3q3ffx~{dRfhÆnYf.dPiMf3º Kº låega3dRf.adhn{f.dGiMftMa3q3jUtMacep´Ùrsfx~´macrszP~Mf.6r|iMrsfxrsdPvsrs.a3t ¹Ca3fx´ma3ac4i À