"The quadratic resource requirements of the attention mechanism are the main roadblock in scaling up transformers to long sequences. This paper replaces the full quadratic attention mechanism by a combination of random attention, window attention, and global attention."

**Lukáš Lánský** @lukas@social.lansky.name · Jul 18, 2020, 18:48

**Lukáš Lánský** @lukas@social.lansky.name · Jul 18, 2020, 18:48

Jul 18, 2020, 18:48

Lukáš Lánský @lukas@social.lansky.name

RT @ares_emu@twitter.com

"Imagine all the extraordinary things we'll be able to do once computers have literally hundreds of CPU cores!"
Developers:

🐦🔗: https://twitter.com/ares_emu/status/1284328853171605508

f0e8becb61492e85.mp4

**Lukáš Lánský** @lukas@social.lansky.name · Jul 14, 2020, 10:01

**Lukáš Lánský** @lukas@social.lansky.name · Jul 14, 2020, 10:01

Jul 14, 2020, 10:01

Lukáš Lánský @lukas@social.lansky.name

"Current prediction markets are so bad in so many different ways that it simply is not surprising for people to know better than them, and it often is not possible for people to make money from knowing better."

https://www.lesswrong.com/posts/c3iQryHA4tnAvPZEv/limits-of-prediction-markets

**Lukáš Lánský** @lukas@social.lansky.name · Jul 13, 2020, 07:27

**Lukáš Lánský** @lukas@social.lansky.name · Jul 13, 2020, 07:27

Jul 13, 2020, 07:27

Lukáš Lánský @lukas@social.lansky.name

RT @components_ai@twitter.com

Give GPT-3 a color scale and an emoji. Get back new scales based on color of the emoji. HOW DOES IT KNOW.

violet: [
'#2d1832',
'#502b5a',
'#753f83',
#8e4c9e',
'#9f5bb0',
'#b683c3',
#c9a2d2',
'#dbc1e1',
'#ebddee',
'#f7f1f8'
],
🍑: [

🐦🔗: https://twitter.com/components_ai/status/1282379087412174848

**Lukáš Lánský** @lukas@social.lansky.name · Jul 04, 2020, 08:26

**Lukáš Lánský** @lukas@social.lansky.name · Jul 04, 2020, 08:26

Jul 04, 2020, 08:26

Lukáš Lánský @lukas@social.lansky.name

RT @quocleix@twitter.com

A surprising result: We found that smooth activation functions are better than ReLU for adversarial training and can lead to substantial improvements in adversarial robustness.
http://arxiv.org/abs/2006.14536

🐦🔗: https://twitter.com/quocleix/status/1277668867066621952

01bbe05fc2aa2e2a.jpg

**Lukáš Lánský** @lukas@social.lansky.name · Jun 24, 2020, 18:48

**Lukáš Lánský** @lukas@social.lansky.name · Jun 24, 2020, 18:48

Jun 24, 2020, 18:48

Lukáš Lánský @lukas@social.lansky.name

RT @citnaj@twitter.com

So yeah I'm really happy about this new DeOldify model actually......

🐦🔗: https://twitter.com/citnaj/status/1275688161369300993

73e459e2fc72a6c8.jpg

**Lukáš Lánský** @lukas@social.lansky.name · Jun 23, 2020, 14:30

**Lukáš Lánský** @lukas@social.lansky.name · Jun 23, 2020, 14:30

Jun 23, 2020, 14:30

Lukáš Lánský @lukas@social.lansky.name

RT @tylercowen@twitter.com

I am Scott Alexander.

🐦🔗: https://twitter.com/tylercowen/status/1275409192149450752

**Lukáš Lánský** @lukas@social.lansky.name · Jun 20, 2020, 11:10

**Lukáš Lánský** @lukas@social.lansky.name · Jun 20, 2020, 11:10

Jun 20, 2020, 11:10

Lukáš Lánský @lukas@social.lansky.name

RT @JanelleCShane@twitter.com

When an AI is trained on words, weird things can happen to the physical domain.

I asked the @OpenAI@twitter.com API about horses.
https://aiweirdness.com/post/621186154843324416/all-your-questions-answered

🐦🔗: https://twitter.com/JanelleCShane/status/1273296527662841856

f45077464d788a2a.jpg

**Lukáš Lánský** @lukas@social.lansky.name · Jun 18, 2020, 12:45

**Lukáš Lánský** @lukas@social.lansky.name · Jun 18, 2020, 12:45

Jun 18, 2020, 12:45

Lukáš Lánský @lukas@social.lansky.name

RT @grhmc@twitter.com

C programmers are like

🐦🔗: https://twitter.com/grhmc/status/1273062119362252802

75331910f36df1bb.jpg

**Lukáš Lánský** @lukas@social.lansky.name · Jun 17, 2020, 21:40

**Lukáš Lánský** @lukas@social.lansky.name · Jun 17, 2020, 21:40

Jun 17, 2020, 21:40

Lukáš Lánský @lukas@social.lansky.name

https://arxiv.org/abs/1911.03584

"Specifically, we prove that a multi-head self-attention layer with sufficient number of heads is at least as expressive as any convolutional layer. Our numerical experiments then show that self-attention layers attend to pixel-grid patterns similarly to CNN layers, corroborating our analysis."

**Lukáš Lánský** @lukas@social.lansky.name · Jun 17, 2020, 19:17

**Lukáš Lánský** @lukas@social.lansky.name · Jun 17, 2020, 19:17

Jun 17, 2020, 19:17

Lukáš Lánský @lukas@social.lansky.name

https://openai.com/blog/image-gpt/

"We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting."

**Lukáš Lánský** @lukas@social.lansky.name · Jun 16, 2020, 20:57

**Lukáš Lánský** @lukas@social.lansky.name · Jun 16, 2020, 20:57

Jun 16, 2020, 20:57

Lukáš Lánský @lukas@social.lansky.name

RT @ahejlsberg@twitter.com

Variadic tuple types are coming to TypeScript 4.0. @typescript@twitter.com #TypeScript https://github.com/microsoft/TypeScript/pull/39094

🐦🔗: https://twitter.com/ahejlsberg/status/1272986860957003788

**Lukáš Lánský** · Jun 14, 2020, 06:45

Lukáš Lánský boosted

**Hacker News 50** @hn50@social.lansky.name · Jun 14, 2020, 06:45

Jun 14, 2020, 06:45

Hacker News 50 @hn50@social.lansky.name

The rise of embarrassingly parallel serverless compute

Link: https://davidwells.io/blog/rise-of-embarrassingly-parallel-serverless-compute
Discussion: https://news.ycombinator.com/item?id=23506637

**Lukáš Lánský** · Jun 13, 2020, 03:07

Lukáš Lánský boosted

**muesli** @fribbledom@mastodon.social · Jun 13, 2020, 03:07

Jun 13, 2020, 03:07

muesli @fribbledom@mastodon.social

The Alternative Big O notation:

O(1) = O(yeah)
O(log n) = O(nice)
O(n) = O(k)
O(n²) = O(my)
O(2ⁿ) = O(no)
O(n!) = O(mg!)

**Lukáš Lánský** @lukas@social.lansky.name · Jun 12, 2020, 09:39

**Lukáš Lánský** @lukas@social.lansky.name · Jun 12, 2020, 09:39

Jun 12, 2020, 09:39

Lukáš Lánský @lukas@social.lansky.name

RT @taaiconaut@twitter.com

🐦🔗: https://twitter.com/taaiconaut/status/1271126894872637443

dda96333614d5a43.jpg

**Lukáš Lánský** @lukas@social.lansky.name · Jun 10, 2020, 22:41

**Lukáš Lánský** @lukas@social.lansky.name · Jun 10, 2020, 22:41

Jun 10, 2020, 22:41

Lukáš Lánský @lukas@social.lansky.name

RT @runfaster2000@twitter.com

Announcing .NET 5.0 Preview 5 https://devblogs.microsoft.com/dotnet/announcing-net-5-0-preview-5/

🐦🔗: https://twitter.com/runfaster2000/status/1270778500274720769

**Lukáš Lánský** @lukas@social.lansky.name · May 31, 2020, 10:13

**Lukáš Lánský** @lukas@social.lansky.name · May 31, 2020, 10:13

May 31, 2020, 10:13

Lukáš Lánský @lukas@social.lansky.name

https://www.sivv.io/article/5ecededf46cc9f76d4639b24/In-marginal-decisions,-favour-action-over-inaction

"After following study participants for six months after making their decision, Levitt found that those who had opted for the choice that involved making a change (as opposed to sticking with the status quo) were more satisfied with their decision and generally happier."