Summary:
These preambles are built by ASTUnit and clangd. Previously, preambles
were always stored on disk.
In-memory preambles are routed back to the compiler as virtual files in
a custom VFS.
Interface of ASTUnit does not allow to use in-memory preambles, as
ASTUnit::CodeComplete receives FileManager as a parameter, so we can't
change VFS used by the compiler inside the CodeComplete method.
A follow-up commit will update clangd in clang-tools-extra to use
in-memory preambles.
Reviewers: klimek, sammccall, bkramer
Reviewed By: klimek
Subscribers: ioeric, cfe-commits
Differential Revision: https://reviews.llvm.org/D39842
llvm-svn: 318411
This patch fixes broken preamble-skipping when the preamble region includes a byte order mark (BOM). Previously, parsing would fail if preamble PCH generation was enabled and a BOM was present.
This also fixes preamble invalidation when a BOM appears or disappears. This may seem to be an obscure edge case, but it happens regularly with IDEs that pass buffer overrides that never (or always) have a BOM, yet the underlying file from the initial parse that generated a PCH might (or might not) have a BOM.
I've included a test case for these scenarios.
Differential Revision: https://reviews.llvm.org/D37491
llvm-svn: 313796
Extend the -fmodule-file option to support the [<name>=]<file> value format.
If the name is omitted, then the old semantics is preserved (the module file
is loaded whether needed or not). If the name is specified, then the mapping
is treated as just another prebuilt module search mechanism, similar to
-fprebuilt-module-path, and the module file is only loaded if actually used
(e.g., via import). With one exception: this mapping also overrides module
file references embedded in other modules (which can be useful if module files
are moved/renamed as often happens during remote compilation).
This override semantics requires some extra work: we now store the module name
in addition to the file name in the serialized AST representation.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D35020
llvm-svn: 312220
Extend the -fmodule-file option to support the [<name>=]<file> value format.
If the name is omitted, then the old semantics is preserved (the module file
is loaded whether needed or not). If the name is specified, then the mapping
is treated as just another prebuilt module search mechanism, similar to
-fprebuilt-module-path, and the module file is only loaded if actually used
(e.g., via import). With one exception: this mapping also overrides module
file references embedded in other modules (which can be useful if module files
are moved/renamed as often happens during remote compilation).
This override semantics requires some extra work: we now store the module name
in addition to the file name in the serialized AST representation.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D35020
llvm-svn: 312105
We use this when running a preprocessor-only action on an AST file in order to
avoid paying the runtime cost of loading the extra information.
llvm-svn: 306760
No-one was using this, and it's not meaningful in general -- FrontendActions
can be run on inputs that don't have a corresponding source file. The current
frontend input can be obtained by asking the FrontendAction if any future
action actually needs it.
llvm-svn: 305045
To support this, an optional marker "#pragma clang module contents" is
recognized in module map files, and the rest of the module map file from that
point onwards is treated as the source of the module. Preprocessing a module
map produces the input module followed by the marker and then the preprocessed
contents of the module.
Ignoring line markers, a preprocessed module might look like this:
module A {
header "a.h"
}
#pragma clang module contents
#pragma clang module begin A
// ... a.h ...
#pragma clang module end
The preprocessed output generates line markers, which are not accepted by the
module map parser, so -x c++-module-map-cpp-output should be used to compile
such outputs.
A couple of major parts do not work yet:
1) The files that are listed in the module map must exist on disk, in order to
build the on-disk header -> module lookup table in the PCM file. To fix
this, we need the preprocessed output to track the file size and other stat
information we might use to build the lookup table.
2) Declaration ownership semantics don't work properly yet, since mapping from
a source location to a module relies on mapping from FileIDs to modules,
which we can't do if module transitions can occur in the middle of a file.
llvm-svn: 302309
action to the general FrontendAction infrastructure.
This permits applying -E, -ast-dump, -fsyntax-only, and so on to a module map
compilation. (The -E form is not currently especially useful yet as there's no
good way to take the output and use it to actually build a module.)
In order to support this, -cc1 now accepts -x <lang>-module-map in all cases
where it accepts -x <lang> for a language we can parse (not ir/ast). And for
uniformity, we also accept -x <lang>-header for all such languages (we used
to reject for cuda and renderscript), and -x <lang>-cpp-output for all such
languages (we used to reject for c, cl, and renderscript).
(None of these new alternatives are accepted by the driver yet, so no
user-visible changes.)
llvm-svn: 301610
In r276159, we started to say that a module X is defined in a pch if we specify
-fmodule-name when building the pch. This caused a regression that reports
module X is defined in both pch and pcm if we generate the pch with
-fmodule-name=X and then in a separate clang invocation, we include the pch and
also import X.pcm.
This patch adds an option CompilingPCH similar to CompilingModule. When we use
-fmodule-name=X while building a pch, modular headers in X will be textually
included and the compiler knows that we are not building module X, so we don't
put module X in SUBMODULE_DEFINITION of the pch.
Differential Revision: http://reviews.llvm.org/D28415
llvm-svn: 291465
Sort the headers by name before adding the includes in
collectModuleHeaderIncludes. This makes the include order for building
umbrellas deterministic across different filesystems and also guarantees
that the ASTWriter always dump top headers in the same order.
There's currently no good way to test for this behavior.
rdar://problem/28116411
llvm-svn: 289478
In this mode, there is no need to load any module map and the programmer can
simply use "@import" syntax to load the module directly from a prebuilt
module path. When loading from prebuilt module path, we don't support
rebuilding of the module files and we ignore compatible configuration
mismatches.
rdar://27290316
Differential Revision: http://reviews.llvm.org/D23125
llvm-svn: 279096
With PCH+Module, sometimes compiler gives a hard error:
Module file ‘<some-file path>.pcm' is out of date and needs to be rebuilt
This happens when we have a pch importing a module and the module gets
overwritten by another compiler instance after we build the pch (one example is
that both compiler instances hash to the same pcm file but use different
diagnostic options). When we try to load the pch later on, the compiler notices
that the imported module is out of date (modification date, size do not match)
but it can't handle this out of date pcm (i.e it does not know how to rebuild
the pch).
This commit introduces a new command line option so for PCH + module, we can
turn on this option and if two compiler instances only differ in diagnostic
options, the latter instance will not invalidate the original pcm.
rdar://26675801
Differential Revision: http://reviews.llvm.org/D22773
llvm-svn: 276769
This changes the CompilerInstance::createOutputFile function to return
a std::unique_ptr<llvm::raw_ostream>, rather than an llvm::raw_ostream
implicitly owned by the CompilerInstance. This in most cases required that
I move ownership of the output stream to the relevant ASTConsumer.
The motivation for this change is to allow BackendConsumer to be a client
of interfaces such as D20268 which take ownership of the output stream.
Differential Revision: http://reviews.llvm.org/D21537
llvm-svn: 275507
This is to allow distributed build systems, that do not preserve time stamps, to use PCH files.
Second and last part of the patch proposed at:
Differential Revision: http://reviews.llvm.org/D20867
llvm-svn: 275267
Summary:
Create a new Frontend LangOpt to specify the renderscript language. It
is enabled by the "-x renderscript" option from the driver.
Add a "kernel" function attribute only for RenderScript (an "ignored
attribute" warning is generated otherwise).
Make the NativeHalfType and NativeHalfArgsAndReturns LangOpts be implied
by the RenderScript LangOpt.
Reviewers: rsmith
Subscribers: cfe-commits, srhines
Differential Revision: http://reviews.llvm.org/D21198
llvm-svn: 272342
Clang performs directory walk while searching headers inside modules by
using the ::sys::fs instead of ::vfs. This prevents any code that uses
the VFS (e.g, reproducer scripts) to actually find such headers, since
the VFS will never be searched for those.
Change these places to use vfs::recursive_directory_iterator and
vfs::directory_iterator instead.
Differential Revision: http://reviews.llvm.org/D20266
rdar://problem/25880368
llvm-svn: 269661
Summary:
There virtual destructor can't be found and cause a compilation error
on a shared build.
To repro: [Release + Shared]
```
-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
```
Which produce this error:
```
tools/clang/unittests/Tooling/CMakeFiles/ToolingTests.dir/ToolingTest.cpp.o: In function `clang::tooling::newFrontendActionFactory_CreatesFrontendActionFactoryFromType_Test::TestBody()':
ToolingTest.cpp:(.text._ZN5clang7tooling66newFrontendActionFactory_CreatesFrontendActionFactoryFromType_Test8TestBodyEv+0x49): undefined reference to `clang::SyntaxOnlyAction::~SyntaxOnlyAction()'
```
Reviewers: alexfh
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D20218
llvm-svn: 269334
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.
This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.
llvm-svn: 261372
building a module. Prior to this change, the private header's content would
only be included if the header were included by another header in the same
module. If not (if the private header is only used by the .cc files of the
module, or is included from outside the module via -Wno-private-header),
a #include of that file would be silently ignored.
llvm-svn: 257222
than reusing the "overridden buffer" mechanism. This will allow us to make
embedded files and overridden files behave differently in future.
llvm-svn: 254121
This flag causes all files that were read by the compilation to be embedded
into a produced module file. This is useful for distributed build systems that
use an include scanning system to determine which files are "needed" by a
compilation, and only provide those files to remote compilation workers. Since
using a module can require any file that is part of that module (or anything it
transitively includes), files that are not found by an include scanner can be
required in a regular build using explicit modules. With this flag, only files
that are actually referenced by transitively-#included files are required to be
present on the build machine.
llvm-svn: 253950
Introduce the notion of a module file extension, which introduces
additional information into a module file at the time it is built that
can then be queried when the module file is read. Module file
extensions are identified by a block name (which must be unique to the
extension) and can write any bitstream records into their own
extension block within the module file. When a module file is loaded,
any extension blocks are matched up with module file extension
readers, that are per-module-file and are given access to the input
bitstream.
Note that module file extensions can only be introduced by
programmatic clients that have access to the CompilerInvocation. There
is only one such extension at the moment, which is used for testing
the module file extension harness. As a future direction, one could
imagine allowing the plugin mechanism to introduce new module file
extensions.
llvm-svn: 251955
file in the .pcm files. This allows a smaller set of files to be sent to a
remote build worker when building with explicit modules (for instance, module
map files need not be sent along with the corresponding precompiled modules).
This doesn't actually make the embedded files visible to header search, so
it's not useful as a packaging format for public header files.
llvm-svn: 245028
- introduces a new cc1 option -fmodule-format=[raw,obj]
with 'raw' being the default
- supports arbitrary module container formats that libclang is agnostic to
- adds the format to the module hash to avoid collisions
- splits the old PCHContainerOperations into PCHContainerWriter and
a PCHContainerReader.
Thanks to Richard Smith for reviewing this patch!
llvm-svn: 242499
Any extra features from -fmodule-feature are part of the module hash and
need to get validated on load. Also print them with -module-file-info.
llvm-svn: 240433
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers and clang modules). The default implementation is
RawPCHContainerOperations, which uses a flat file for the output.
The main application for this interface will be an
ObjectFilePCHContainerOperations implementation that uses LLVM to
wrap the module in an ELF/Mach-O/COFF container to store debug info
alongside the AST.
rdar://problem/20091852
llvm-svn: 240225
LLVM can now detect if a fd is seekable on windows.
Original commit message:
Actually check if lseek works instead of using a filename based heuristic.
llvm-svn: 234738
