179 Commits

Author SHA1 Message Date
Argyrios Kyrtzidis
abff5f1271 Improve Lexer::getImmediateMacroName to take into account inner macros
of macro arguments.

For "MAC1( MAC2(foo) )" and location of 'foo' token it would return
"MAC1" instead of "MAC2".

llvm-svn: 148704
2012-01-23 16:58:33 +00:00
Argyrios Kyrtzidis
85e7671b71 Enhance Lexer::makeFileCharRange to check for ranges inside a macro argument
expansion, in which case it returns a file range in the location where the
argument was spelled.

llvm-svn: 148551
2012-01-20 16:52:43 +00:00
Argyrios Kyrtzidis
7838a2bffb Introduce Lexer::getSourceText() that returns a string for the source
that the given source range encompasses.

llvm-svn: 148481
2012-01-19 15:59:19 +00:00
Argyrios Kyrtzidis
a99e02d019 Introduce Lexer::makeFileCharRange() that accepts a token source range
and returns a character range with file locations.

llvm-svn: 148480
2012-01-19 15:59:14 +00:00
Argyrios Kyrtzidis
1b07c344b4 For Lexer's isAt[Start/End]OfMacroExpansion add an out parameter for the macro
start/end location.

It is commonly needed after calling the function; with this way we avoid
recalculating it.

llvm-svn: 148479
2012-01-19 15:59:08 +00:00
Anna Zaks
1bea4bf590 Refactor: Pull getImmediateMacroName() out of DiagnosticRenderer and
into Lexer and Preprocessor; making it widely available.

llvm-svn: 148410
2012-01-18 20:17:16 +00:00
Chandler Carruth
5b15a9be6a Two variables had been added for an assert, but their values were
re-computed rather than the variables be re-used just after the assert.
Just use the variables since we have them already. Fixes an unused
variable warning.

Also fix an 80-column violation.

llvm-svn: 148212
2012-01-15 09:03:45 +00:00
Argyrios Kyrtzidis
8a26c4de64 In Lexer::getCharAndSizeSlow[NoWarn] if we come up against
\<newline><newline>

don't consume the second newline.

Thanks to David Blaikie for pointing out the crash!

llvm-svn: 147138
2011-12-22 04:38:07 +00:00
Argyrios Kyrtzidis
e5cdd080ba In Lexer::getCharAndSizeSlow[NoWarn] make sure we don't go over the end of the buffer
when the end of the buffer is immediately after an escaped newline.

Fixes http://llvm.org/PR10153.

llvm-svn: 147091
2011-12-21 20:19:55 +00:00
David Blaikie
68e081d606 Unweaken vtables as per http://llvm.org/docs/CodingStandards.html#ll_virtual_anch
llvm-svn: 146959
2011-12-20 02:48:34 +00:00
Benjamin Kramer
900f1defdd Remove assert from hot code path and add a clarifying comment.
The assert wasn't adding much value but slowed down Release+Asserts builds.

llvm-svn: 145082
2011-11-22 20:39:31 +00:00
Benjamin Kramer
3885737a1b Lexer: Don't throw away the hard work SSE did to find a slash.
We can reuse the information and avoid looping over all the bytes again.

llvm-svn: 145070
2011-11-22 18:56:46 +00:00
Ted Kremenek
a08713ce86 Move about 20 random diagnostics under -W flags. Patch by Ahmed Charles!
llvm-svn: 142284
2011-10-17 21:47:53 +00:00
Richard Smith
acd4d3d52a -Wc++98-compat warnings for the lexer.
This also adds a -Wc++98-compat-pedantic for warning on constructs which would
be diagnosed by -std=c++98 -pedantic (that is, it warns even on C++11 features
which we enable by default, with no warning, in C++98 mode).

llvm-svn: 142034
2011-10-15 01:18:56 +00:00
Douglas Gregor
227c352bae We do parse hexfloats in C++11; make it actually work.
llvm-svn: 141798
2011-10-12 18:51:02 +00:00
Richard Smith
a9e33d44a6 Handle Perforce-style conflict markers like normal conflict markers. Perforce
swaps over the <<<< and >>>> markers, and uses shorter markers than traditional
tools.

llvm-svn: 141751
2011-10-12 00:37:51 +00:00
Abramo Bagnara
e398e60611 Fixed exapnsion range for # and ##.
llvm-svn: 141012
2011-10-03 18:39:03 +00:00
Argyrios Kyrtzidis
e6e67deeed Rename SourceLocation::getFileLocWithOffset -> getLocWithOffset.
It already works (and is useful with) macro locs as well.

llvm-svn: 140057
2011-09-19 20:40:19 +00:00
Francois Pichet
0706d203cf Rename LangOptions::Microsoft to LangOptions::MicrosoftExt to make it clear that this flag must be used only for Microsoft extensions and not emulation; to avoid confusion with the new LangOptions::MicrosoftMode flag.
Many of the code now under LangOptions::MicrosoftExt will eventually be moved under the LangOptions::MicrosoftMode flag.

llvm-svn: 139987
2011-09-17 17:15:52 +00:00
Benjamin Kramer
17ff23c708 Speed up BCPL comment lexing by looking aggressively for newlines and then scannig backwards to see if the newline is escaped.
3% speedup in preprocessing all of clang with -Eonly. Also includes a small testcase for coverage.

llvm-svn: 139116
2011-09-05 07:19:39 +00:00
Benjamin Kramer
dbfb18a0a9 Use the Lexer's definition of whitespace here.
llvm-svn: 139115
2011-09-05 07:19:35 +00:00
Argyrios Kyrtzidis
5cec2aea3f Support code-completion for C++ inline methods and ObjC buffering methods.
Previously we would cut off the source file buffer at the code-completion
point; this impeded code-completion inside C++ inline methods and,
recently, with buffering ObjC methods.

Have the code-completion inserted into the source buffer so that it can
be buffered along with a method body. When we actually hit the code-completion
point the cut-off lexing or parsing.

Fixes rdar://10056932&8319466

llvm-svn: 139086
2011-09-04 03:32:15 +00:00
Argyrios Kyrtzidis
a3deaeeb52 Fix Lexer::ComputePreamble when MaxLines parameter is non-zero.
The function was only counting lines that included tokens and not empty lines,
but MaxLines (mainly initiated to the line where the code-completion point resides)
is a count of overall lines (even empty ones).

llvm-svn: 139085
2011-09-04 03:32:04 +00:00
Douglas Gregor
081425343b Introduce support for a simple module import declaration, which
loads the named module. The syntax itself is intentionally hideous and
will be replaced at some later point with something more
palatable. For now, we're focusing on the semantics:
  - Module imports are handled first by the preprocessor (to get macro
  definitions) and then the same tokens are also handled by the parser
  (to get declarations). If both happen (as in normal compilation),
  the second one is redundant, because we currently have no way to
  hide macros or declarations when loading a module. Chris gets credit
  for this mad-but-workable scheme.
  - The Preprocessor now holds on to a reference to a module loader,
  which is responsible for loading named modules. CompilerInstance is
  the only important module loader: it now knows how to create and
  wire up an AST reader on demand to actually perform the module load.
  - We search for modules in the include path, using the module name
  with the suffix ".pcm" (precompiled module) for the file name. This
  is a temporary hack; we hope to improve the situation in the
  future.

llvm-svn: 138679
2011-08-26 23:56:07 +00:00
Argyrios Kyrtzidis
7aecbc7661 Make Lexer::ComputePreamble accept a LangOptions parameter, otherwise it may be
out-of-sync how a file is compiled. Patch by Matthias Kleine!

llvm-svn: 138580
2011-08-25 20:39:19 +00:00
Argyrios Kyrtzidis
f6a3b0ca4b In Lexer::isAtEndOfMacroExpansion use SourceManager::isInFileID and avoid
the extra SourceManager::getFileID call.

llvm-svn: 138376
2011-08-23 21:02:30 +00:00
Argyrios Kyrtzidis
161868db4c Make Lexer::GetBeginningOfToken able to handle macro arg expansion locations.
llvm-svn: 137795
2011-08-17 00:31:23 +00:00
Craig Topper
54edccafc5 Add support for C++0x raw string literals.
llvm-svn: 137298
2011-08-11 04:06:15 +00:00
Anna Zaks
59a3c80717 Add a utility function to the Lexer, which makes it easier to find a token after the given location. (It is a generalized version of trans::findLocationAfterSemi from ArcMigrate, which will be changed to use the Lexer utility).
llvm-svn: 136268
2011-07-27 21:43:43 +00:00
Douglas Gregor
fb65e592e0 Add support for C++0x unicode string and character literals, from Craig Topper!
llvm-svn: 136210
2011-07-27 05:40:30 +00:00
Chandler Carruth
ee4c1d1298 Migrate 'Instantiation' data and API bits of SLocEntry to 'Expansion'
etc. With this I think essentially all of the SourceManager APIs are
converted. Comments and random other bits of cleanup should be all thats
left.

llvm-svn: 136057
2011-07-26 04:56:51 +00:00
Chandler Carruth
73ee5d7fae Convert InstantiationInfo and much of the related code to ExpansionInfo
and various other 'expansion' based terms. I've tried to reformat where
appropriate and catch as many references in comments but I'm going to do
several more passes. Also I've tried to expand parameter names to be
more clear where appropriate.

llvm-svn: 136056
2011-07-26 04:41:47 +00:00
Chandler Carruth
115b077f30 Rename create(MacroArg)InstantiationLoc to create(MacroArg)ExpansionLoc.
llvm-svn: 136054
2011-07-26 03:03:05 +00:00
Chandler Carruth
ca757587a3 Rename SourceManager::getImmediateInstantiationRange to
getImmediateExpansionRange.

llvm-svn: 135960
2011-07-25 20:52:21 +00:00
Chandler Carruth
6d28d7f2a3 Rename SourceManager::getInstantiationRange to getExpansionRange.
llvm-svn: 135915
2011-07-25 16:56:02 +00:00
Chandler Carruth
35f5320d8e Mechanically rename SourceManager::getInstantiationLoc and
FullSourceLoc::getInstantiationLoc to ...::getExpansionLoc. This is part
of the API and documentation update from 'instantiation' as the term for
macros to 'expansion'.

llvm-svn: 135914
2011-07-25 16:49:02 +00:00
Chris Lattner
0e62c1cc0b remove unneeded llvm:: namespace qualifiers on some core types now that LLVM.h imports
them into the clang namespace.

llvm-svn: 135852
2011-07-23 10:55:15 +00:00
Joerg Sonnenberger
da5d2b761a Spelling
llvm-svn: 135545
2011-07-20 00:14:37 +00:00
Douglas Gregor
925296b4c2 Revamp the SourceManager to separate the representation of parsed
source locations from source locations loaded from an AST/PCH file.

Previously, loading an AST/PCH file involved carefully pre-allocating
space at the beginning of the source manager for the source locations
and FileIDs that correspond to the prefix, and then appending the
source locations/FileIDs used for parsing the remaining translation
unit. This design forced us into loading PCH files early, as a prefix,
whic has become a rather significant limitation.

This patch splits the SourceManager space into two parts: for source
location "addresses", the lower values (growing upward) are used to
describe parsed code, while upper values (growing downward) are used
for source locations loaded from AST/PCH files. Similarly, positive
FileIDs are used to describe parsed code while negative FileIDs are
used to file/macro locations loaded from AST/PCH files. As a result,
we can load PCH/AST files even during parsing, making various
improvemnts in the future possible, e.g., teaching #include <foo.h> to
look for and load <foo.h.gch> if it happens to be already available.

This patch was originally written by Sebastian Redl, then brought
forward to the modern age by Jonathan Turner, and finally
polished/finished by me to be committed.

llvm-svn: 135484
2011-07-19 16:10:42 +00:00
Chandler Carruth
e2c09ebcaa Convert terminology in the Lexer from 'instantiate' and variants to
'expand'. Also update the public API it provides to the new term, and
propagate that update to the various clients.

No functionality changed.

llvm-svn: 135138
2011-07-14 08:20:40 +00:00
Argyrios Kyrtzidis
61c58f7f43 Move SourceManager::isAt[Start/End]OfMacroInstantiation functions to the Lexer, since they depend on it now.
llvm-svn: 134644
2011-07-07 21:54:45 +00:00
Argyrios Kyrtzidis
41fb2d95a3 Make the Preprocessor more memory efficient and improve macro instantiation diagnostics.
When a macro instantiation occurs, reserve a SLocEntry chunk with length the
full length of the macro definition source. Set the spelling location of this chunk
to point to the start of the macro definition and any tokens that are lexed directly
from the macro definition will get a location from this chunk with the appropriate offset.

For any tokens that come from argument expansion, '##' paste operator, etc. have their
instantiation location point at the appropriate place in the instantiated macro definition
(the argument identifier and the '##' token respectively).
This improves macro instantiation diagnostics:

Before:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
        ^~~~
t.c:5:11: note: instantiated from:
int y = M(/);
          ^

After:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
        ^~~~
t.c:3:20: note: instantiated from:
\#define M(op) (foo op 3);
                ~~~ ^  ~
t.c:5:11: note: instantiated from:
int y = M(/);
          ^

The memory savings for a candidate boost library that abuses the preprocessor are:

- 32% less SLocEntries (37M -> 25M)
- 30% reduction in PCH file size (900M -> 635M)
- 50% reduction in memory usage for the SLocEntry table (1.6G -> 800M)

llvm-svn: 134587
2011-07-07 03:40:34 +00:00
Argyrios Kyrtzidis
2cfce18645 Allow Lexer::getLocForEndOfToken to return the location just passed the macro instantiation
if the location given points at the last token of the macro instantiation.

Fixes rdar://9045701.

llvm-svn: 133804
2011-06-24 17:58:59 +00:00
Eli Friedman
86a5101c27 Don't strlen() every file before parsing it.
llvm-svn: 131132
2011-05-10 17:11:21 +00:00
Chris Lattner
57540c5be0 fix a bunch of comment typos found by codespell. Patch by
Luis Felipe Strano Moraes!

llvm-svn: 129559
2011-04-15 05:22:18 +00:00
Richard Smith
f7b6202e6c Implement C++0x [lex.pptoken]p3's handling of <::.
llvm-svn: 129525
2011-04-14 18:36:27 +00:00
Eric Christopher
7f36a79ee9 Eat the UTF-8 BOM at the beginning of a file since it's ignored anyhow.
Nom Nom Nom.

Patch by Anton Korobeynikov!

llvm-svn: 129174
2011-04-09 00:01:04 +00:00
John McCall
75ca6d72c2 Fix getLocForEndOfToken to not double-count spurious internal characters
within a token, like trigraphs and escaped newlines.               
Patch by Marcin Kowalczyk!

llvm-svn: 128978
2011-04-06 01:50:22 +00:00
Daniel Dunbar
1057f86d0e Lexer: Add extremely limited support for -traditional-cpp, ignoring BCPL
comments.

llvm-svn: 127910
2011-03-18 21:23:38 +00:00
John McCall
462c055d85 Fix my earlier commit to work with escaped newlines and leave breadcrumbs
in case we want to make a world where we can check intermediate instantiations
for this kind of breadcrumb.

llvm-svn: 127221
2011-03-08 07:59:04 +00:00