Built for deep thinking. Built for serious writing.

InLine.

AI, Always InLine.

Download for macOS More platforms

Attention Is All You Need

DocumentLiteratureImages

Revision author: InLine

Attention Is All You Need

Vaswani et al. · 2017 · arXiv:1706.03762 ↗
Short excerpt & reading notes

“We propose a new simple network architecture, the Transformer, based solely on attention mechanisms” […]

Attention computation

Attention compares each query with a set of keys to determine how much weight to assign to the corresponding values. These scores are normalized before the values are combined:

\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\!\left(\frac{QK^{T}}{\sqrt{d_k}}\right)V

(1)

~~The matrices Q, K, and V represent queries, keys, and values.~~The matrices Q, K, and V denote the query, key, and value matrices. The key dimension determines the scaling factor in the denominator. Scaling moderates the dot products before softmax, helping preserve useful gradients.

Reading note

~~Each head looks at the same information.~~Each head can capture different relationships. In practice, separate learned projections allow attention heads to represent different relationships. Their outputs are concatenated and projected to produce a combined representation.

Together, these projections provide complementary views of the same sequence.

Knowledge reuse

Your sources, shared with AI.

Live collaboration

One space, instant response.

Controlled revisions

Every change, traceable.

01 / Writing workspace

Workspace, built for collaboration.

Let AI see what you see.

InLine AI

Hello! What can I do for you?

Add the Attention formula here.

This section already explains how attention scores are computed. I will add the scaled dot-product form after the colon and submit it as a pending revision.

Attention Is All You Need

Tell me what you need…

02 / InLine AI

InLine AI, one tap away

Let AI think how you think.

InLine AI

Hello! What can I do for you?

Attention Is All You Need

Tell me what you need…

03 / LaTeX projects

LaTeX support, real-time compilation

Let AI write how you write.

04 / Document Support

Word documents, seamless transition

Let AI do what you do.

Wattention-is-all-you-need.docx

FileHomeInsertLayoutReviewView

Attention Is All You Need

Vaswani et al. · 2017 · Short excerpt & reading notes

“We propose a new simple network architecture, the Transformer, based solely on attention mechanisms” […]

Attention computation

Attention compares each query with a set of keys to determine how much weight to assign to the corresponding values. These scores are normalized before the values are combined:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ)V (1)

The matrices Q, K, and V represent queries, keys, and values. The key dimension determines the scaling factor in the denominator. Scaling moderates the dot products before softmax, helping preserve useful gradients.

Reading note

Each head looks at the same information. In practice, separate learned projections allow attention heads to represent different relationships. Their outputs are concatenated and projected to produce a combined representation.

Wattention-is-all-you-need_approved.docx

FileHomeInsertLayoutReviewView

Attention Is All You Need

Vaswani et al. · 2017 · Short excerpt & reading notes

“We propose a new simple network architecture, the Transformer, based solely on attention mechanisms” […]

Attention computation

Attention compares each query with a set of keys to determine how much weight to assign to the corresponding values. These scores are normalized before the values are combined:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ)V (1)

The matrices Q, K, and V ~~represent queries, keys, and values~~denote the query, key, and value matrices. The key dimension determines the scaling factor in the denominator. Scaling moderates the dot products before softmax, helping preserve useful gradients.

Reading note

Waiting for a document

More features

Beyond writing,
there is more.

From source gathering to delivery, keep everyday work in one place.

Vast sources, instant search

Built-in document search with synonym-aware and complex queries, shared with your Agent

legitimate peripheral participation

Synonymslegitimate peripheral participation

8 files show 16 matcheslegitimate peripheral participation

Social Network Analysis: A Framework for Identifying Communities in Higher Education Online Learning

Shazia K. Jan, Panos Vlachopoulos, 2019Technology, Knowledge and Learning

P.4

Mutual engagement refers to interactions between individuals which occur within a network and lead to rhythms of participation and non-participation (Wenger et al. 2009). The process of legitimate peripheral participation or identity development (Lave and Wenger 1991)…

P.4

Structural changes in a network over time would depict these learning trajectories which signify legitimate peripheral participation, identity formation and, learning—the critical components of a CoP…

P.11

Core-periphery structure A clear core-periphery structure representing legitimate peripheral participation No core-periphery structure representing equal participation

View 3 more matches

Literature corpus, always within reach

Organize sources, excerpts, images, and other materials, ready to bring into context whenever you write.

All references

10 items

Search title, author, journal, text, tags, or use

Latest

Select a reference to view its saved excerpts.

ReferenceAuthorJournalYearExcerptsTags

Bowling Alone but Online Together: Social Capital in E-Communities James K. Scott, T. G. Johnson 50 Years of Community Development Vol II 20050

social capitalonline communitiescommunity development

The influence of social capital on knowledge creation in online health communities Jing Zhao, Sejin Ha, R. Widdows Journal of Special Topics in Information Technology and Management 20150

social capitalknowledge creationonline communities

Knowledge sharing behavior in learning online communities: A social capital perspective Guoxin Li, Y.-J. Li 2010 IEEE International Conference on Management of Innovation & Technology 20100

social capitalknowledge sharingonline learning community

Tacit knowledge in online learning: community, identity, and social capital Murat Oztok — 20130

social capitalonline learningtacit knowledgecommunity

Local first, documents secure

Files, sources, and API keys all stay on your device.

Local spaceInLine Device

Local only

Documents & filesDOCX · Markdown · PDF Local

Sources & notesText, notes, and images Local

API keysPlugins and model services Local

Rich plugins, shared context

InLine AI, local agents, and source plugins share the same writing context.

Agent6

InLine AIBuilt-in Agent

ClaudeLocal Agent

ChatGPT (Codex)Local Agent

DeepSeek HarnessAgent runtime

ZcodeAgent client

WorkbuddyWorkflow assistant

Sources & services3

ZoteroLocal library

ObsidianLocal notes

Semantic ScholarLiterature search

Leave a little room for your next piece of work.

Download for macOS

InLine.

AI, Always InLine.

Attention Is All You Need

Attention computation

Reading note

Your sources, shared with AI.

One space, instant response.

Every change, traceable.

Workspace, built for collaboration.

InLine AI, one tap away

LaTeX support, real-time compilation

Word documents, seamless transition

Beyond writing, there is more.

All references

Start with an idea.

Beyond writing,
there is more.