Currently the implementation of __VA_OPT__ will treat the concatenation of a
non-placemaker token and placemaker token as a placemaker token which is not
correct. This will fix the implementation and treat the result as a
non-placemaker token.
This fixes: https://github.com/llvm/llvm-project/issues/60268
Differential Revision: https://reviews.llvm.org/D142604
This patch is a reland of 74e4f778cf16cbf7163b5c6de6027a43f5e9169f and
f83347b0bedb22ea676861c8e4e2ed9c31371ade with the removed 50 trick back.
The magic number 50 was removed in D134942, as a behavior change for
performance reason.
While it reduces the number of SLocEntry, it increases the usage of
SourceLocation address space usage, which is critical for compiling
large TU.
This fixes a regression caused in D134942 -- clang failed to compile one of
our internal files, complaining the file is too large to process because clang
runs out of source location space (we spend 40% more address space!)
Differential Revision: https://reviews.llvm.org/D136539
This reverts commit 74e4f778cf16cbf7163b5c6de6027a43f5e9169f, which caused a
~40% increase in SLoc address space utilization on certain cases. See
https://reviews.llvm.org/D136539#3877872.
This reverts commit f83347b0bedb22ea676861c8e4e2ed9c31371ade. This is necessary
to revert 74e4f778cf16cbf7163b5c6de6027a43f5e9169f, which caused a ~40% increase
in SLoc address space utilization on certain cases. See
https://reviews.llvm.org/D136539#3877872.
The code falls back to the pre-2011 partition-file-id solution (see for
[details](https://reviews.llvm.org/D20401#3823476)).
This patch simplifies/rewrites the code based on the partition-based-on-file-id
idea. The new implementation is optimized by reducing the number of
calling getFileID (~40% drop).
Despite the huge drop of getFileID, this is a marignal improvment on
speed (becase the number of calling non-cached getFileID is roughly
the same). It removes the evaluation-order performance gap between gcc-built-clang
and clang-built-clang.
SemaExpr.cpp:
- before: 315063 SLocEntries, FileID scans: 388230 linear, 1393437 binary. 458893 cache hits, 672299 getFileID calls
- after: 313494 SLocEntries, FileID scans: 397525 linear, 1451890 binary, 176714 cache hits, 397144 getFileID calls
FindTarget.cpp:
- before: 279984 SLocEntries, FileID scans: 361926 linear, 1275930 binary, 436072 cache hits, 632150 getFileID calls
- after: 278426 SLocEntries, FileID scans: 371279 linear, 1333963 binary, 153705 cache hits, 356814 getFileID calls
Differential Revision: https://reviews.llvm.org/D134942
This is part of a patch series working towards the ability to make
SourceLocation into a 64-bit type to handle larger translation units.
NFC: this patch introduces typedefs for the integer type used by
SourceLocation and makes all the boring changes to use the typedefs
everywhere, but for the moment, they are unconditionally defined to
uint32_t.
Patch originally by Mikhail Maltsev.
Reviewed By: tmatheson
Differential Revision: https://reviews.llvm.org/D105492
This reverts commit f4537935dcdbf390c863591cf556e76c3abab9c1.
This reverts commit b43c26d036dcbf7a6881f39e4434cf059364022a.
This GNU and MSVC extension turns out to be very popular. Most projects
are not using C++20, so cannot use the new __VA_OPT__ feature to be
standards conformant. The other workaround, using -std=gnu*, enables too
many language extensions and isn't viable.
Until there is a way for users to get the behavior provided by the
`, ## __VA_ARGS__` extension in the -std=c++17 and earlier language
modes, we need to revert this.
As noted in D91913, MSVC implements the GNU behavior for
, ## __VA_ARGS__ as well. Do the same when `-fms-compatibility` is used.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D95392
The GNU token paste extension that removes the comma in , ## __VA_ARGS__
conflicts with C99/C++11's requirements when a variadic macro has no
named parameters: according to the standard, an invocation as FOO()
gives it a single empty argument, and concatenation of anything with an
empty argument is well-defined. For this reason, the GNU extension was
already disabled in C99 standard-conforming mode. It was not yet
disabled in C++11 standard-conforming mode.
The associated comment suggested that GCC keeps this extension enabled
in C90/C++03 standard-conforming mode, but it actually does not, so
rather than adding a check for C++ language version, this change simply
removes the check for C language version.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D91913
Summary:
The cache recorded the wrong expansion location for all but the first
stringization. It seems uncommon to stringize the same macro argument
multiple times, so this cache doesn't seem that important.
Fixes PR39942
Reviewers: vsk, rsmith
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65428
llvm-svn: 367337
Summary:
By adding a hook to consume all tokens produced by the preprocessor.
The intention of this change is to make it possible to consume the
expanded tokens without re-runnig the preprocessor with minimal changes
to the preprocessor and minimal performance penalty when preprocessing
without recording the tokens.
The added hook is very low-level and reconstructing the expanded token
stream requires more work in the client code, the actual algorithm to
collect the tokens using this hook can be found in the follow-up change.
Reviewers: rsmith
Reviewed By: rsmith
Subscribers: eraman, nemanjai, kbarton, jsji, riccibruno, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D59885
llvm-svn: 361007
* __VA_OPT__ is expanded if the *expanded* __VA_ARGS__ is non-empty,
not if the original argument contained no tokens.
* Placemarkers at the start and end of __VA_OPT__ are retained just
long enough to paste them with adjacent ## operators. We never paste
"across" a discarded placemarker.
llvm-svn: 359964
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This is similar to the LLVM change https://reviews.llvm.org/D46290.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46320
llvm-svn: 331834
When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
llvm-svn: 331155
Found via codespell -q 3 -I ../clang-whitelist.txt
Where whitelist consists of:
archtype
cas
classs
checkk
compres
definit
frome
iff
inteval
ith
lod
methode
nd
optin
ot
pres
statics
te
thru
Patch by luzpaz! (This is a subset of D44188 that applies cleanly with a few
files that have dubious fixes reverted.)
Differential revision: https://reviews.llvm.org/D44188
llvm-svn: 329399
This patch implements an extension to the preprocessor:
__VA_OPT__(contents) --> which expands into its contents if variadic arguments are supplied to the parent macro, or behaves as an empty token if none.
- Currently this feature is only enabled for C++2a (this could be enabled, with some careful tweaks, for other dialects with the appropriate extension or compatibility warnings)
- The patch was reviewed here: https://reviews.llvm.org/D35782 and asides from the above (and moving some of the definition and expansion recognition logic into the corresponding state machines), I believe I incorporated all of Richard's suggestions.
A few technicalities (most of which were clarified through private correspondence between rsmith, hubert and thomas) are worth mentioning. Given:
#define F(a,...) a #__VA_OPT__(a ## a) a ## __VA_OPT__(__VA_ARGS__)
- The call F(,) Does not supply any tokens for the variadic arguments and hence VA_OPT behaves as a placeholder.
- When expanding VA_OPT (for e.g. F(,1) token pasting occurs eagerly within its contents if the contents need to be stringified.
- A hash or a hashhash prior to VA_OPT does not inhibit expansion of arguments if they are the first token within VA_OPT.
- When a variadic argument is supplied, argument substitution occurs within the contents as does stringification - and these resulting tokens are inserted back into the macro expansions token stream just prior to the entire stream being rescanned and concatenated.
See wg21.link/P0306 for further details on the feature.
Acknowledgment: This patch would have been poorer if not for Richard Smith's usual thoughtful analysis and feedback.
llvm-svn: 315840
- it made the bots v angry!
I'm not exactly sure why the assertion doesn't hold - if anyone has any insight - would appreciate it.
Thanks!
llvm-svn: 314748
In passing:
- change the name of the function to pasteTokens c/w coding standards
- rename CurToken to CurTokenIdx (since it is not the token, but the index)
- add doxygen comments to document some of pasteTokens' functionality
- use parameter names different from the data member names.
This will be useful for implementing __VA_OPT__ (https://reviews.llvm.org/D35782#inline-322587)
llvm-svn: 314747
- MacroArgs already knows the maximum number of arguments that can be supplied to the macro. No need to pass MacroInfo (information about the macro definition) to the call to getPreExpArgument (which by the way might benefit from being called getExpandedArgument() ?) for it to compute the number of arguments.
llvm-svn: 314593
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
I will also directly update some extra clang tooling that is broken by the change from Argument to Parameter.
Hopefully the bots will stay appeased.
Thanks!
llvm-svn: 308190
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
Thanks!
llvm-svn: 308157
The whitespace should come from the argument name in the macro
expansion, rather than from the token passed to the macro (same as it
does when not pasting).
Added a new test case for the change in behavior to stringize_space.c.
FileCheck'ized macro_paste_commaext.c, tweaked the test case, and
added a comment; no behavioral change to this test.
Differential Revision: https://reviews.llvm.org/D30427
llvm-svn: 302195
While in the area, also change some unsigned variables to size_t, and
introduce an LLVM_FALLTHROUGH instead of a comment stating that.
Differential Revision: http://reviews.llvm.org/D25982
llvm-svn: 285193
The lexer sets the end location of macro arguments incorrectly *if*,
while merging consecutive args to fit into a single SLocEntry, it finds
args which come from different macro files.
Fix the issue by using separate SLocEntries in this situation.
This fixes a code coverage crasher (rdar://problem/26181005). Because
the lexer reported end locations for certain macro args incorrectly, we
would generate bogus coverage mappings with negative line offsets.
Reviewed-by: akyrtzi
Differential Revision: http://reviews.llvm.org/D20401
llvm-svn: 270160
Summary:
This fixes PR25875. When the trailing comma in a macro argument list is
elided, we need to treat it similarly to the case where a variadic macro
misses one actual argument.
Reviewers: rnk, rsmith
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D15670
llvm-svn: 258530
first token of the expansion, don't forget to copy the "is at the start of a
line" token (which is always false, as newlines cannot appear within a macro
body); otherwise, stringizing the result can insert spurious whitespace.
llvm-svn: 257863
In public MS headers for XAudio, clang would fail to generate a valid UUID due to the UUID components being combined with the '-' UUID separators. Clang would attempting to recover but would preserve the leading whitespace from the tokens after each failed paste leading to spaces creeping into the UUID and causing an error in the __declspace(uuid()) parsing.
Reference: Microsoft DirectX SDK (June 2010)\Include\XAudio2.h(51)
Resolves http://llvm.org/pr23071
llvm-svn: 235186
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.
There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while
SmallString do not. We may wish to change this, but it may introduce
ambiguity.
llvm-svn: 232622
Clang should form a wide string literal from L#macro_arg in a function-like macro in -fms-compatibility mode.
Fix for http://llvm.org/PR9984.
Differential Revision: http://reviews.llvm.org/D6604
llvm-svn: 224228
We would crash because we used operator[] to access past the end of a
SmallString. This occured because our token had length zero.
Instead, form the pointer using .data() and arithmetic. This is safe
because this forms a one-past-the-end pointer and it is only used to
compare with another one-past-the-end pointer.
This fixes PR21379.
llvm-svn: 220614
