https://arxiv.org/abs/1911.03584
"Specifically, we prove that a multi-head self-attention layer with sufficient number of heads is at least as expressive as any convolutional layer. Our numerical experiments then show that self-attention layers attend to pixel-grid patterns similarly to CNN layers, corroborating our analysis."
https://openai.com/blog/image-gpt/
"We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting."
RT @ahejlsberg@twitter.com
Variadic tuple types are coming to TypeScript 4.0. @typescript@twitter.com #TypeScript https://github.com/microsoft/TypeScript/pull/39094
🐦🔗: https://twitter.com/ahejlsberg/status/1272986860957003788
The rise of embarrassingly parallel serverless compute
Link: https://davidwells.io/blog/rise-of-embarrassingly-parallel-serverless-compute
Discussion: https://news.ycombinator.com/item?id=23506637
RT @runfaster2000@twitter.com
Announcing .NET 5.0 Preview 5 https://devblogs.microsoft.com/dotnet/announcing-net-5-0-preview-5/
🐦🔗: https://twitter.com/runfaster2000/status/1270778500274720769
"After following study participants for six months after making their decision, Levitt found that those who had opted for the choice that involved making a change (as opposed to sticking with the status quo) were more satisfied with their decision and generally happier."
RT @ak92501@twitter.com
Language Models are Few-Shot Learners
pdf: https://arxiv.org/pdf/2005.14165.pdf
abs: https://arxiv.org/abs/2005.14165
github: https://github.com/openai/gpt-3
https://dl.acm.org/doi/pdf/10.1145/3317550.3321445
Unikernels: The Next Stage of Linux’s Dominance
https://distill.pub/2020/grand-tour/
Visualizing Neural Networks with the Grand Tour
RT @OpenAI@twitter.com
Since 2012, the amount of compute for training to AlexNet-level performance on ImageNet has been decreasing exponentially — halving every 16 months, in total a 44x improvement.
By contrast, Moore's Law would only have yielded an 11x cost improvement: https://openai.com/blog/ai-and-efficiency/
RT @_cartermp@twitter.com
Introducing #csharp Source Generators, try it out with the latest #dotnet 5 preview! https://devblogs.microsoft.com/dotnet/introducing-c-source-generators/
🐦🔗: https://twitter.com/_cartermp/status/1255605990885437441
I'm a Prague based software engineer. This is my personal account.