When querying the index during an ObjC protocol name lookup for code
completion, we should only suggest ObjC protocols.
Differential Revision: https://reviews.llvm.org/D127125
Previously, clangd would filter completions only on the first part of
the selector (first typed chunk) instead of all remaining selector
fragments (all typed chunks).
Differential Revision: https://reviews.llvm.org/D124637
Add support for concepts and requires expression in the clang index.
Genarate USRs for concepts.
Also change how `RecursiveASTVisitor` handles return type requirement in
requires expressions. The new code unpacks the synthetic template parameter
list used for storing the actual expression. This simplifies
implementation of the indexing. No code seems to depend on the original
traversal anyway and the synthesized template parameter list is easily
accessible from inside the requires expression if needed.
Add tests in the clangd codebase.
Fixes https://github.com/clangd/clangd/issues/1103.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D124441
Auto-generated patch based on clang-tidy readability-identifier-naming.
Only some manual cleanup for `extern "C"` declarations and a GTest change was required.
I'm not sure if this cleanup is actually very useful. It cleans up clang-tidy findings to the number of warnings from clang-tidy should be lower. Since it was easy to do and required only little cleanup I thought I'd upload it for discussion.
One pattern that keeps recurring: Test **matchers** are also supposed to start with a lowercase letter as per LLVM convention. However GTest naming convention for matchers start with upper case. I would propose to keep stay consistent with the GTest convention there. However that would imply a lot of `//NOLINT` throughout these files.
To re-product this patch run:
```
run-clang-tidy -checks="-*,readability-identifier-naming" -fix -format ./clang-tools-extra/clangd
```
To convert the macro names, I was using this script with some manual cleanup afterwards:
https://gist.github.com/ChristianKuehnel/a01cc4362b07c58281554ab46235a077
Differential Revision: https://reviews.llvm.org/D115634
This is a workaround (adding a newline to the eof) in clangd to avoid the code
completion crash, see https://github.com/clangd/clangd/issues/332.
In principle, this is a clang bug, we should fix it in clang, but it is not
trivial.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D117456
C++ member function bodies (including ctor initializers) are first captured
into a buffer and then parsed after the class is complete. (This allows
members to be referenced even if declared later).
When the boundary of the function body cannot be established, its buffer is
discarded and late-parsing never happens (it would surely fail).
For code completion this is the wrong tradeoff: the point of the parse is to
generate completions as a side-effect.
Today, when the ctor body wasn't typed yet there are no init list completions.
With this patch we parse such an init-list if it contains the completion point.
There's one caveat: the parser has to decide where to resume parsing members
after a broken init list. Often the first clear recovery point is *after* the
next member, so that member is missing from completion/signature help etc. e.g.
struct S {
S() m //<- completion here
int maaa;
int mbbb;
}
Here "int maaa;" is treated as part of the init list, so "maaa" is not available
as a completion. Maybe in future indentation can be used to recognize that
this is a separate member, not part of the init list.
Differential Revision: https://reviews.llvm.org/D116294
The "parameter list" is the list of fields which should be initialized.
We introduce a new OverloadCandidate kind for this.
It starts to become harder for CC consumers to handle all the cases for
params, so I added some extra APIs on OverloadCandidate to abstract them.
Includes some basic support for designated initializers.
The same aggregate signature is shown, the current arg jumps after the
one you just initialized. This follows C99 semantics for mixed
designated/positional initializers (which clang supports in C++ as an extension)
and is also a useful prompt for C++ as C++ designated initializers must be
in order.
Related bugs:
- https://github.com/clangd/clangd/issues/965
- https://github.com/clangd/clangd/issues/306
Differential Revision: https://reviews.llvm.org/D116326
Implementation is based on the "expected type" as used for
designated-initializers in braced init lists. This means it can deduce the type
in some cases where it's not written:
void foo(Widget);
foo({ /*help here*/ });
Only basic constructor calls are in scope of this patch, excluded are:
- aggregate initialization (no help is offered for aggregates)
- initializer_list initialization (no help is offered for these constructors)
Fixes https://github.com/clangd/clangd/issues/306
Differential Revision: https://reviews.llvm.org/D116317
Provide signature while typing template arguments: Foo< ^here >
Here the parameters are e.g. "typename x", and the result type is e.g.
"struct" (class template) or "int" (variable template) or "bool (std::string)"
(function template).
Multiple overloads are possible when a template name is used for several
overloaded function templates.
Fixes https://github.com/clangd/clangd/issues/299
Differential Revision: https://reviews.llvm.org/D116352
This unifies the behaviour we have in code completion item
documentations and signaturehelp. Providing better line wrapping and detection
of inline code blocks in comments to be renedered appropriately in markdown.
Differential Revision: https://reviews.llvm.org/D115442
This covers both C-style variadic functions and template variadic w/
parameter packs.
Previously we would return no signatures when working with template
variadic functions once activeParameter reached the position of the
parameter pack (except when it was the only param, then we'd still
show it when no arguments were given). With this commit, we now show
signathure help correctly.
Additionally, this commit fixes the activeParameter value in LSP output
of clangd in the presence of variadic functions (both kinds). LSP does
not allow the activeParamter to be higher than the number of parameters
in the active signature. With "..." or parameter pack being just one
argument, for all but first argument passed to "..." we'd report
incorrect activeParameter value. Clients such as VSCode would then treat
it as 0, as suggested in the spec) and highlight the wrong parameter.
In the future, we should add support for per-signature activeParamter
value, which exists in LSP since 3.16.0. This is not part of this
commit.
Differential Revision: https://reviews.llvm.org/D111318
For example, if you have:
void foo(int bar);
foo(/*^
it should auto-complete to "bar=".
Because Sema callbacks for code completion in comments happen before we
have an AST we need to cheat in clangd by detecting completion on /*
before, moving cursor back by two characters, then running a simplified
verion of SignatureHelp to extract argument name(s) from possible
overloads.
Differential Revision: https://reviews.llvm.org/D110823
Fixes https://github.com/clangd/clangd/issues/819
SourceLocation of macros change when a header file is included above it. This is not checked when creating a PreamblePatch, resulting in reusing previously built preamble with an incorrect source location for the macro in the example test case.
This patch stores the SourceLocation in the struct TextualPPDirective so that it gets checked when comparing old vs new preambles.
Also creates a preamble patch for code completion parsing so that clangd does not crash when following the example test case with a large file.
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D108045
... to the one of signature hints.
In particular, completion items now also carry annotations, which client
code might be interested in.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D107365
They are already provided by Sema, deserializing from preamble if need
be. Moreover category names are meaningless outside interface/implementation
context, hence they were only causing noise.
Differential Revision: https://reviews.llvm.org/D104540
When completing an Objective-C method declaration by name, we need to
preserve the leading text as a `qualifier` so we insert it properly
before the first typed text chunk.
Differential Revision: https://reviews.llvm.org/D100798
Having nested macros in the C code could cause clangd to fail an assert in clang::Preprocessor::setLoadedMacroDirective() and crash.
#1 0x00000000007ace30 PrintStackTraceSignalHandler(void*) /qdelacru/llvm-project/llvm/lib/Support/Unix/Signals.inc:632:1
#2 0x00000000007aaded llvm::sys::RunSignalHandlers() /qdelacru/llvm-project/llvm/lib/Support/Signals.cpp:76:20
#3 0x00000000007ac7c1 SignalHandler(int) /qdelacru/llvm-project/llvm/lib/Support/Unix/Signals.inc:407:1
#4 0x00007f096604db20 __restore_rt (/lib64/libpthread.so.0+0x12b20)
#5 0x00007f0964b307ff raise (/lib64/libc.so.6+0x377ff)
#6 0x00007f0964b1ac35 abort (/lib64/libc.so.6+0x21c35)
#7 0x00007f0964b1ab09 _nl_load_domain.cold.0 (/lib64/libc.so.6+0x21b09)
#8 0x00007f0964b28de6 (/lib64/libc.so.6+0x2fde6)
#9 0x0000000001004d1a clang::Preprocessor::setLoadedMacroDirective(clang::IdentifierInfo*, clang::MacroDirective*, clang::MacroDirective*) /qdelacru/llvm-project/clang/lib/Lex/PPMacroExpansion.cpp:116:5
An example of the code that causes the assert failure:
```
...
```
During code completion in clangd, the macros will be loaded in loadMainFilePreambleMacros() by iterating over the macro names and calling PreambleIdentifiers->get(). Since these macro names are store in a StringSet (has StringMap underlying container), the order of the iterator is not guaranteed to be same as the order seen in the source code.
When clangd is trying to resolve nested macros it sometimes attempts to load them out of order which causes a macro to be stored twice. In the example above, ECHO2 macro gets resolved first, but since it uses another macro that has not been resolved it will try to resolve/store that as well. Now there are two MacroDirectives stored in the Preprocessor, ECHO and ECHO2. When clangd tries to load the next macro, ECHO, the preprocessor fails an assert in clang::Preprocessor::setLoadedMacroDirective() because there is already a MacroDirective stored for that macro name.
In this diff, I check if the macro is already inside the IdentifierTable and if it is skip it so that it is not resolved twice.
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D101870
Somewhat surprisingly, signature help is emitted as a side-effect of
computing the expected type of a function argument.
The reason is that both actions require enumerating the possible
function signatures and running partial overload resolution, and doing
this twice would be wasteful and complicated.
Change #1: document this, it's subtle :-)
However, sometimes we need to compute the expected type without having
reached the code completion cursor yet - in particular to allow
completion of designators.
eb4ab3358cd4dc834a761191b5531b38114f7b13 did this but introduced a
regression - it emits signature help in the wrong location as a side-effect.
Change #2: only emit signature help if the code completion cursor was reached.
Currently there is PP.isCodeCompletionReached(), but we can't use it
because it's set *after* running code completion.
It'd be nice to set this implicitly when the completion token is lexed,
but ConsumeCodeCompletionToken() makes this complicated.
Change #3: call cutOffParsing() *first* when seeing a completion token.
After this, the fact that the Sema::Produce*SignatureHelp() functions
are even more confusing, as they only sometimes do that.
I don't want to rename them in this patch as it's another large
mechanical change, but we should soon.
Change #4: prepare to rename ProduceSignatureHelp() to GuessArgumentType() etc.
Differential Revision: https://reviews.llvm.org/D98488
CodeCompletionContext::Kind has 36 Kinds. The completion model currently
only handles categorical features of 32 cardinality.
Changing the datatype to uint64_t will solve the problem.
This reverts commit 438b5bb05a429d697674088d400e4800c1108658.
This makes code completion use a Decision Forest based ranking algorithm to rank
completion candidates. [Esitmated 6% accuracy boost]. This was
previously hidden behind the flag --ranking-model=decision_forest. This
patch makes it the default ranking algorithm.
Note: this is a generic model, not specialized for any particular
project. clangd does not collect or upload data to train code completion.
Also treat Keywords separately as they are not recorded by the training set generator.
Differential Revision: https://reviews.llvm.org/D96353
Now, given `template <typename T> foo() {}` when user types `fo^<int>()` the
completion snippet will not contain `<int>()`.
Also, when the next token is opening parenthesis (`(`) and completion snippet
contains template argument list, it is still emitted.
This patch complements D81380.
Related issue: https://github.com/clangd/clangd/issues/387
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D89870
Today, inside a template, you can get completion for:
Foo<T> t;
t.^
t has dependent type Foo<T>, and we use the primary template to find its members.
However we also want this to work:
t.foo.bar().^
The type of t.foo.bar() is DependentTy, so we attempt to resolve using similar
heuristics (e.g. primary template).
Differential Revision: https://reviews.llvm.org/D96376
Current indexes merge logic skip data from the static index if the file is in the dynamic index, but sometimes the dynamic index does not contain references (e.g. preamble (dynamic) index vs background (static) index).
This problem is masked with the fact, that the preamble index file list consists of file URI's and other indexes file lists consist of file paths.
This patch introduces the index contents (symbols, references, etc.), which makes indexes merge more flexible and makes it able to use URI's for the index file list.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D94952
This patch only introduces new signals but does not use their value
in scoring a CC candidate. Usage of these signals in CC ranking in both
heiristics and ML model will be introduced in later patches.
Differential Revision: https://reviews.llvm.org/D94473
This patch fixes the following problem:
- open a file with references to the symbol `Foo`
- remove all references to `Foo` (from the dynamic index).
- `MergedIndex::refs()` result will contain positions of removed references (from the static index).
The idea of this patch is to keep a set of files which were used during index build inside the index.
Thus at processing the static index references we can check if the file of processing reference is a part of the dynamic index or not.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D93393
This can happen when, for example, merging results from an external
index that generates IncludeHeaders with full URI rather than just
literal include.
Differential Revision: https://reviews.llvm.org/D92494
With every incremental change, one needs to check-in new model upstream.
This also significantly increases the size of the git repo with every
new model.
Testing and comparing the old and previous model is also not possible as
we run only a single model at any point.
One solution is to have a "staging" decision forest which can be
injected into clangd without pushing it to upstream. Compare the
performance of the staging model with the live model. After a couple of
enhancements have been done to staging model, we can then replace the
live model upstream with the staging model. This reduces upstream churn
and also allows us to compare models with current baseline model.
This is done by having a callback in CodeCompleteOptions which is called
only when we want to use a decision forest ranking model. This allows us
to inject different completion model internally.
Differential Revision: https://reviews.llvm.org/D90014
so that we could start experiment for C.
Previously, these flags in clangd were only meaningful for C++. We need
to flip them for C, this patch repurpose these flags.
- if true, just set it.
- if false, just respect the value in clang.
this would allow us to keep flags on for C++, and optionally flip them on for C.
Differential Revision: https://reviews.llvm.org/D89233
By default clangd will score a code completion item using heuristics model.
Scoring can be done by Decision Forest model by passing `--ranking_model=decision_forest` to
clangd.
Features omitted from the model:
- `NameMatch` is excluded because the final score must be multiplicative in `NameMatch` to allow rescoring by the editor.
- `NeedsFixIts` is excluded because the generating dataset that needs 'fixits' is non-trivial.
There are multiple ways (heuristics) to combine the above two features with the prediction of the DF:
- `NeedsFixIts` is used as is with a penalty of `0.5`.
Various alternatives of combining NameMatch `N` and Decision forest Prediction `P`
- N * scale(P, 0, 1): Linearly scale the output of model to range [0, 1]
- N * a^P:
- More natural: Prediction of each Decision Tree can be considered as a multiplicative boost (like NameMatch)
- Ordering is independent of the absolute value of P. Order of two items is proportional to `a^{difference in model prediction score}`. Higher `a` gives higher weightage to model output as compared to NameMatch score.
Baseline MRR = 0.619
MRR for various combinations:
N * P = 0.6346, advantage%=2.5768
N * 1.1^P = 0.6600, advantage%=6.6853
N * **1.2**^P = 0.6669, advantage%=**7.8005**
N * **1.3**^P = 0.6668, advantage%=**7.7795**
N * **1.4**^P = 0.6659, advantage%=**7.6270**
N * 1.5^P = 0.6646, advantage%=7.4200
N * 1.6^P = 0.6636, advantage%=7.2671
N * 1.7^P = 0.6629, advantage%=7.1450
N * 2^P = 0.6612, advantage%=6.8673
N * 2.5^P = 0.6598, advantage%=6.6491
N * 3^P = 0.6590, advantage%=6.5242
N * scaled[0, 1] = 0.6465, advantage%=4.5054
Differential Revision: https://reviews.llvm.org/D88281
We intend to replace heuristics based code completion ranking with a Decision Forest Model.
This patch introduces a format for representing the model and an inference runtime that is code-generated at build time.
- Forest.json contains all the trees as an array of trees.
- Features.json describes the features to be used.
- Codegen file takes the above two files and generates CompletionModel containing Feature struct and corresponding Evaluate function.
The Evaluate function maps a feature to a real number describing the relevance of this candidate.
- The codegen is part of build system and these files are generated at build time.
- Proposes a way to test the generated runtime using a test model.
- Replicates the model structure in unittests.
- unittest tests both the test model (for correct tree traversal) and the real model (for sanity).
This reverts commit 549e55b3d5634870aa9d42135f51ad46a6a0e347.
Summary:
[WIP]
- Proposes a json format for representing Random Forest model.
- Proposes a way to test the generated runtime using a test model.
TODO:
- Add generated source code snippet for easier review.
- Fix unused label warning.
- Figure out required using declarations for CATEGORICAL columns from Features.json.
- Necessary Google3 internal modifications for blaze before landing.
- Add documentation for format of the model.
- Document more.
Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D83814
Summary:
Clangd uses FSProvider to get threadsafe views into file systems. This
patch changes naming to make that more explicit.
Depends on D81920
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D81998
Reland https://reviews.llvm.org/D76696
All known crashes have been fixed, another attemption.
We have rolled out this to all internal users for a while, didn't see
big issues, we consider it is stable enough.
Reviewed By: sammccall
Subscribers: rsmith, hubert.reinterpretcast, ebevhan, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D78350
Summary:
Prevent a second pair of parenthesis from being added when there already is one
right after cursor.
Related issue and more context: https://github.com/clangd/clangd/issues/387
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D81380
