Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.
- Move definition ASTFileSignature to Basic/Module.h so Module and
ASTSourceDescriptor can use it.
- Change the signature from uint64_t to std::array<uint32_t,5>.
- Stop using (saving/reading) the size and modification time of PCM
files when there is a valid SIGNATURE.
- Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
and other records that shouldn't affect the hash. Because implicit
modules reuses the same file for multiple levels of -Werror, this
includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.
This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system. This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).
Design and original patch by Manman Ren!
llvm-svn: 297655
Initialize fields directly in header. Note that the ModuleManager field is an
IntrusiveRefCntPtr, so there's no need for explicit initialization.
llvm-svn: 293863
Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was expecting the function returned a unique_ptr,
but instead it returned a raw pointer - introducing a leak.
Thanks Aleksey!
This reapplies r291184, reverted in r291249.
llvm-svn: 291270
Merge all VFS mapped files inside -ivfsoverlay inputs into the vfs
overlay provided by the crash reproducer. This is the last missing piece
to allow crash reproducers to fully work with user frameworks; when
combined with headermaps, it allows clang to find additional frameworks.
rdar://problem/27913709
llvm-svn: 290326
Collect the necessary input PCH files.
Do not try to validate the AST before copying it out because if the
crash is in this path, we won't be able to collect it. Instead only
check if it's a file containg an AST.
rdar://problem/27913709
llvm-svn: 289460
Include headermaps (.hmap files) in the .cache directory and
add VFS entries. All headermaps are known after HeaderSearch
setup, collect them right after.
rdar://problem/27913709
llvm-svn: 289360
Recover better from an incompatible .pcm file being provided by -fmodule-file=. We try to include the headers of the module textually in this case, still enforcing the modules semantic rules. In order to make that work, we need to still track that we're entering and leaving the module. Also, if the module was also marked as unavailable (perhaps because it was missing a file), we shouldn't mark the module unavailable -- we don't need the module to be complete if we're going to enter it textually.
llvm-svn: 288741
This reverts commit r288449.
I believe that this is currently faulty wrt. modules being imported
inside namespaces. Adding these lines to the new test:
namespace n {
#include "foo.h"
}
Makes it break with
fatal error: import of module 'M' appears within namespace 'n'
However, I believe it should fail with
error: redundant #include of module 'M' appears within namespace 'n'
I have tracked this down to us now inserting a tok::annot_module_begin
instead of a tok::annot_module_include in
Preprocessor::HandleIncludeDirective() and then later in
Parser::parseMisplacedModuleImport(), we hit the code path for
tok::annot_module_begin, which doesn't set FromInclude of
checkModuleImportContext to true (thus leading to the "wrong"
diagnostic).
llvm-svn: 288626
We try to include the headers of the module textually in this case, still
enforcing the modules semantic rules. In order to make that work, we need to
still track that we're entering and leaving the module. Also, if the module was
also marked as unavailable (perhaps because it was missing a file), we
shouldn't mark the module unavailable -- we don't need the module to be
complete if we're going to enter it textually.
llvm-svn: 288449
Summary:
This used to work because system headers are found in a (somewhat)
predictable set of locations on Linux. But this is not the case on
MacOS; without this change, we don't look in the right places for our
headers when doing device-side compilation on Mac.
Reviewers: tra
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D26776
llvm-svn: 287286
This can be used to append alternative typo corrections to an existing diag.
include-fixer can use it to suggest includes to be added.
Differential Revision: https://reviews.llvm.org/D26745
llvm-svn: 287128
Summary:
SetVector already used DenseSet, but SmallSetVector used std::set. This
leads to surprising performance differences. Moreover, it means that
the set of key types accepted by SetVector and SmallSetVector are
quite different!
In order to make this change, we had to convert some callsites that used
SmallSetVector<std::string, N> to use SmallSetVector<CachedHashString, N>
instead.
Reviewers: timshen
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D25648
llvm-svn: 284887
This option behaves in a similar spirit as -save-temps and writes
internal llvm statistics in json format to a file.
Differential Revision: https://reviews.llvm.org/D24820
llvm-svn: 282426
In this mode, there is no need to load any module map and the programmer can
simply use "@import" syntax to load the module directly from a prebuilt
module path. When loading from prebuilt module path, we don't support
rebuilding of the module files and we ignore compatible configuration
mismatches.
rdar://27290316
Differential Revision: http://reviews.llvm.org/D23125
llvm-svn: 279096
Adjust target features for amdgcn target when -cl-denorms-are-zero is set.
Denormal support is controlled by feature strings fp32-denormals fp64-denormals in amdgcn target. If -cl-denorms-are-zero is not set and the command line does not set fp32/64-denormals feature string, +fp32-denormals +fp64-denormals will be on for GPU's supporting them.
A new virtual function virtual void TargetInfo::adjustTargetOptions(const CodeGenOptions &CGOpts, TargetOptions &TargetOpts) const is introduced to allow adjusting target option by codegen option.
Differential Revision: https://reviews.llvm.org/D22815
llvm-svn: 278151
This changes the CompilerInstance::createOutputFile function to return
a std::unique_ptr<llvm::raw_ostream>, rather than an llvm::raw_ostream
implicitly owned by the CompilerInstance. This in most cases required that
I move ownership of the output stream to the relevant ASTConsumer.
The motivation for this change is to allow BackendConsumer to be a client
of interfaces such as D20268 which take ownership of the output stream.
Differential Revision: http://reviews.llvm.org/D21537
llvm-svn: 275507
Summary:
Host and device types must match, otherwise when we pass values back and
forth between the host and device, we will get the wrong result.
This patch makes NVPTXTargetInfo inherit most of its type information
from the host's target info.
Reviewers: rsmith
Subscribers: cfe-commits, jhen, tra
Differential Revision: http://reviews.llvm.org/D19346
llvm-svn: 268131
Revert the two changes to thread CodeGenOptions into the TargetInfo allocation
and to fix the layering violation by moving CodeGenOptions into Basic.
Code Generation is arguably not particularly "basic". This addresses Richard's
post-commit review comments. This change purely does the mechanical revert and
will be followed up with an alternate approach to thread the desired information
into TargetInfo.
llvm-svn: 265806
This threads CodeGenOptions into the TargetInfo hierarchy. This is motivated by
ARM which can change some target information based on the EABI selected
(-meabi). Similar options exist for other platforms (e.g. MIPS) and thus is
generally useful. NFC.
llvm-svn: 265640
- Make ModuleDependencyCollector use the DependencyCollector interface
- Move some methods from ModuleDependencyListener to ModuleDependencyCollector
in order to share common functionality with other future possible
callbacks.
llvm-svn: 264808
Instead of putting the /Yc header into ExtraDeps, give DependencyOutputOptions
a dedicated field for /Yc mode, and let HeaderIncludesCallback hang on to the
full DependencyOutputOptions object, not just ExtraDeps.
Reverts parts of r263352 that are now no longer needed.
llvm-svn: 264182
-H in gcc mode doesn't print -include headers, but they are included in
depfiles written by MMD and friends. Since /showIncludes is what's used instead
of depfiles, printing /FI there seems important (and matches cl.exe).
Instead of giving HeaderIncludeGen more options, just switch on ShowAllHeaders
in clang-cl mode and let clang::InitializePreprocessor() not put -include flags
in the <command line> block. This changes the behavior of -E slightly, and it
removes the <command line> flag from the output triggered by setting the
obscure CC_PRINT_HEADERS=1 env var to true while running clang. Both of these
seem ok to change.
http://reviews.llvm.org/D18401
llvm-svn: 264174
To make this work, delay printing of ExtraDeps in HeaderIncludesCallback a bit,
so that it happens after CompilerInstance::InitializeSourceManager() has run.
General /FI arguments are still missing from /showIncludes output, this still
needs to be fixed.
llvm-svn: 263352
In the gcc precompiled header model, one explicitly runs clang with `-x
c++-header` on a .h file to produce a gch file, and then includes the header
with `-include foo.h` and if a .gch file exists for that header it gets used.
This is documented at
http://clang.llvm.org/docs/UsersManual.html#precompiled-headers
cl.exe's model is fairly different, and controlled by the two flags /Yc and
/Yu. A pch file is generated as a side effect of a regular compilation when
/Ycheader.h is passed. While the compilation is running, the compiler keeps
track of #include lines in the main translation unit and writes everything up
to an `#include "header.h"` line into a pch file. Conversely, /Yuheader.h tells
the compiler to skip all code in the main TU up to and including `#include
"header.h"` and instead load header.pch. (It's also possible to use /Yc and /Yu
without an argument, in that case a `#pragma hrdstop` takes the role of
controlling the point where pch ends and real code begins.)
This patch implements limited support for this in that it requires the pch
header to be passed as a /FI force include flag – with this restriction,
it can be implemented almost completely in the driver with fairly small amounts
of code. For /Yu, this is trivial, and for /Yc a separate pch action is added
that runs before the actual compilation. After r261774, the first failing
command makes a compilation stop – this means if the pch fails to build the
main compilation won't run, which is what we want. However, in /fallback builds
we need to run the main compilation even if the pch build fails so that the
main compilation's fallback can run. To achieve this, add a ForceSuccessCommand
that pretends that the pch build always succeeded in /fallback builds (the main
compilation will then fail to open the pch and run the fallback cl.exe
invocation).
If /Yc /Yu are used in a setup that clang-cl doesn't implement yet, clang-cl
will now emit a "not implemented yet; flag ignored" warning that can be
disabled using -Wno-clang-cl-pch.
Since clang-cl doesn't yet serialize some important things (most notably
`pragma comment(lib, ...)`, this feature is disabled by default and only
enabled by an internal driver flag. Once it's more stable, this internal flag
will disappear.
(The default stdafx.h setup passes stdafx.h as explicit argument to /Yc but not
as /FI – instead every single TU has to `#include <stdafx.h>` as first thing it
does. Implementing support for this should be possible with the approach in
this patch with minimal frontend changes by passing a --stop-at / --start-at
flag from the driver to the frontend. This is left for a follow-up. I don't
think we ever want to support `#pragma hdrstop`, and supporting it with this
approach isn't easy: This approach relies on the driver knowing the pch
filename in advance, and `#pragma hdrstop(out.pch)` can set the output
filename, so the driver can't know about it in advance.)
clang-cl now also honors /Fp and puts pch files in the same spot that cl.exe
would put them, but the pch file format is of course incompatible. This has
ramifications on /fallback, so /Yc /Yu aren't passed through to cl.exe in
/fallback builds.
http://reviews.llvm.org/D17695
llvm-svn: 262420
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.
This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.
llvm-svn: 261372
we can't load that file due to a configuration mismatch, and implicit module
building is disabled, and the user turns off the error-by-default warning for
that situation, then fall back to textual inclusion for the module rather than
giving an error if any of its headers are included.
llvm-svn: 252114
Introduce the notion of a module file extension, which introduces
additional information into a module file at the time it is built that
can then be queried when the module file is read. Module file
extensions are identified by a block name (which must be unique to the
extension) and can write any bitstream records into their own
extension block within the module file. When a module file is loaded,
any extension blocks are matched up with module file extension
readers, that are per-module-file and are given access to the input
bitstream.
Note that module file extensions can only be introduced by
programmatic clients that have access to the CompilerInvocation. There
is only one such extension at the moment, which is used for testing
the module file extension harness. As a future direction, one could
imagine allowing the plugin mechanism to introduce new module file
extensions.
llvm-svn: 251955
via -fmodule-file= to be turned off; in that case, just include the relevant
files textually. This allows module files to be unconditionally passed to all
compile actions via CXXFLAGS, and to be ignored for rules that specify custom
incompatible flags.
llvm-svn: 250577
* adds -aux-triple option to specify target triple
* propagates aux target info to AST context and Preprocessor
* pulls in target specific preprocessor macros.
* pulls in target-specific builtins from aux target.
* sets appropriate host or device attribute on builtins.
Differential Revision: http://reviews.llvm.org/D12917
llvm-svn: 248299
