94 Commits

Author SHA1 Message Date
Puyan Lotfi
5a40cd5b50 [llvm-objcopy] Dwarf .debug section compression support (zlib, zlib-gnu).
Usage:

  llvm-objcopy --compress-debug-sections=zlib foo.o
  llvm-objcopy --compress-debug-sections=zlib-gnu foo.o

  In both cases the debug section contents is compressed with zlib. In the GNU
  style case the header is the "ZLIB" magic string followed by the uint64 big-
  endian decompressed size. In the non-GNU mode the header is the
  Elf(32|64)_Chdr.

  Decompression support is coming soon.


  Differential Revision: https://reviews.llvm.org/D49678

llvm-svn: 341342
2018-09-03 22:25:56 +00:00
Jordan Rupprecht
cf67633e66 [llvm-objcopy] Add support for -I binary -B <arch>.
Summary:
The -I (--input-target) and -B (--binary-architecture) flags exist but are currently silently ignored. This adds support for -I binary for architectures i386, x86-64 (and alias i386:x86-64), arm, aarch64, sparc, and ppc (powerpc:common64). This is largely based on D41687.

This is done by implementing an additional subclass of Reader, BinaryReader, which works by interpreting the input file as contents for .data field, sets up a synthetic header, and adds additional sections/symbols (e.g. _binary__tmp_data_txt_start).

Reviewers: jakehehrlich, alexshap, jhenderson, javed.absar

Reviewed By: jhenderson

Subscribers: jyknight, nemanjai, kbarton, fedor.sergeev, jrtc27, kristof.beyls, paulsemel, llvm-commits

Differential Revision: https://reviews.llvm.org/D50343

llvm-svn: 340070
2018-08-17 18:51:11 +00:00
Jordan Rupprecht
de965ea87b [llvm-objcopy] NFC: consistently use typename ELFT::<X> definitions in headers
llvm-svn: 339448
2018-08-10 16:25:58 +00:00
Jordan Rupprecht
88ed5e59bd [llvm-objcopy] NFC: Add some color to error()
llvm-svn: 339404
2018-08-09 22:52:03 +00:00
Paul Semel
a42dec7a1b [llvm-objcopy] Add --dump-section
Differential Revision: https://reviews.llvm.org/D49979

llvm-svn: 339358
2018-08-09 17:05:21 +00:00
Puyan Lotfi
97604b4f09 [NFC] clang-format cleanup of a couple files in llvm-objcopy.
llvm-svn: 338752
2018-08-02 18:16:52 +00:00
Puyan Lotfi
0f5d5fae93 [NFC][llvm-objcopy] Cleanup namespace usage in llvm-objcopy.
Nest any classes not used outside of a file into anon. Nest any classes used
across files in llvm-objcopy into namespace llvm::objcopy.


Differential Revision: https://reviews.llvm.org/D49449

llvm-svn: 337337
2018-07-18 00:10:51 +00:00
Puyan Lotfi
c4846a551e [NFC][llvm-objcopy] Make helper functions static
Anywhere in tools/llvm-objcopy where functions or classes are not referenced
outside of a given file, we change things to make the function or class static
or put inside an anonymous namespace.

llvm-svn: 337220
2018-07-16 22:17:05 +00:00
Jake Ehrlich
c7f8ac7896 [llvm-objcopy] Add support for large indexes
This patch is an update of an older patch that never landed
(see here: https://reviews.llvm.org/D42516)

Recently various users have run into this issue and it just 100%
has to be solved at this point. The main difference in this patch
is that I use gunzip instead of unzip which should hopefully allow
tests to pass. Please review this as if it is a new patch however.
I found some issues along the way and made some minor modifications.

The binary used in this patch for testing (a zip file to make it small)
can be found here:
https://drive.google.com/file/d/1UjsnTO9edLttZibbr-2T1bJl92KEQFAO/view?usp=sharing

Differential Revision: https://reviews.llvm.org/D49206

llvm-svn: 337204
2018-07-16 19:48:52 +00:00
Alexander Shaposhnikov
42b5ef0269 [llvm-objcopy] Add support for static libraries
This diff adds support for handling static libraries 
to llvm-objcopy and llvm-strip.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D48413

llvm-svn: 336455
2018-07-06 17:51:03 +00:00
Alexander Shaposhnikov
58cb1973de [llvm-objcopy] Remove unused field from Object
The class Object contains std::shared_ptr<MemoryBuffer> OwnedData
which is not used anywhere. Besides avoiding two stage initialization 
the motivation to remove it comes from the plan to add (currently missing) support 
for static libraries.
NFC.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D47855

llvm-svn: 334217
2018-06-07 19:41:42 +00:00
Paul Semel
46201fb7bc [llvm-objcopy] Fix null symbol handling
This fixes the bug where strip-all option was
leading to a malformed outputted ELF file.

Differential Revision: https://reviews.llvm.org/D47414

llvm-svn: 333772
2018-06-01 16:19:46 +00:00
Peter Collingbourne
1651ac13be llvm-objcopy: Set sh_link to 0 on unrecognized symtab-linked sections.
Per discussion on the generic-abi mailing list:
https://groups.google.com/forum/#!topic/generic-abi/MPr8TVtnVn4

An object file manipulation tool must either write out a symbol
table with the same number of entries as the original symbol table
and in the same order, or if this is impossible, refuse to operate
on the object file if it has unrecognized sections that are linked
to the symtab section. However, existing tools (namely GNU strip,
GNU objcopy and ld.{bfd,gold,lld} -r) do not comply with this at
present: they change symbol table indexes and set sh_link to 0 on
the unrecognized symtab-linked sections.

We intend to use the latter as a (temporary) signal that a tool has
operated on a proposed new symtab-linked section and invalidated the
symbol table indexes. However, llvm-objcopy currently keeps sh_link
pointing to the new symtab section. This patch changes llvm-objcopy
to set sh_link to 0 to match the behaviour of the other tools.

Differential Revision: https://reviews.llvm.org/D47404

llvm-svn: 333581
2018-05-30 19:30:39 +00:00
Paul Semel
99dda0bab8 [llvm-objcopy] Add --strip-unneeded option
Differential Revision: https://reviews.llvm.org/D46896

llvm-svn: 333267
2018-05-25 11:01:25 +00:00
Paul Semel
31a212d694 Revert "[llvm-objcopy] Add --strip-unneeded option"
There is a use after free I didn't see. Need to investigate.

This reverts commit f7624abeb1f0d012309baf2e78cf2499fbfe5e5f.

llvm-svn: 332925
2018-05-22 01:04:36 +00:00
Paul Semel
040df77ed6 [llvm-objcopy] Add --strip-unneeded option
This option removes symbols that are not needed by relocations.

Differential Revision: https://reviews.llvm.org/D46896

llvm-svn: 332915
2018-05-21 22:50:32 +00:00
Paul Semel
4246a462a3 [llvm-objcopy] Add --strip-symbol (-N) option
llvm-svn: 331924
2018-05-09 21:36:54 +00:00
Paul Semel
41695f8e73 [llvm-objcopy] Add --discard-all (-x) option
llvm-svn: 331400
2018-05-02 20:19:22 +00:00
Alexander Shaposhnikov
40e9bdf9af [llvm-objcopy] Implement --redefine-sym option
This diff implements --redefine-sym option 
for changing the name of a symbol.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D46029

llvm-svn: 330973
2018-04-26 18:28:17 +00:00
Alexander Shaposhnikov
52db4335b3 [llvm-objcopy] Fix sh_link
This diff fixes sh_link for various types of sections 
(i.e. for SHT_ARM_EXIDX, SHT_HASH). In particular, this change enables us
to use llvm-objcopy with clang -gsplit-dwarf for the target android-arm.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D45851

llvm-svn: 330478
2018-04-20 20:46:04 +00:00
Alexander Shaposhnikov
6ecc6e62c1 [llvm-objcopy] Implement support for section groups
This diff adds support for SHT_GROUP sections to llvm-objcopy.
Some sections are interrelated and comprise a group.
For example, a definition of an inline function might require, 
in addition to the section containing its instructions, 
a read-only data section containing literals referenced inside the function.
A section of the type SHT_GROUP contains the indices of the group members,
therefore, it needs to be updated whenever the indices change.
Similarly, the fields sh_link, sh_info should be recalculated as well.

[Resubmit r328012 with the proper handling of endianness]

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D43996

llvm-svn: 328143
2018-03-21 19:53:44 +00:00
Alexander Shaposhnikov
3b24ed7f21 [llvm-objcopy] Revert r328012
Temporarily revert r328012 (since it broke down the big-endian bots),
will resubmit an updated version later.

llvm-svn: 328024
2018-03-20 19:46:00 +00:00
Alexander Shaposhnikov
43b8acdfd7 [llvm-objcopy] Implement support for section groups
This diff adds support for SHT_GROUP sections to llvm-objcopy.
Some sections are interrelated and comprise a group.
For example, a definition of an inline function might require, 
in addition to the section containing its instructions, 
a read-only data section containing literals referenced inside the function.
A section of the type SHT_GROUP contains the indices of the group members,
therefore, it needs to be updated whenever the indices change.
Similarly, the fields sh_link, sh_info should be recalculated as well.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D43996

llvm-svn: 328012
2018-03-20 18:20:42 +00:00
Jake Ehrlich
8b831c1d5a Revert "[llvm-objcopy] Add support for large indexes"
Not all build bots have unzip which I used in a test.

This reverts commit 0b1f26d39ea42dd3716b525fbc8c78d8c7bb4479.

llvm-svn: 326941
2018-03-07 20:33:02 +00:00
Jake Ehrlich
0a151bd6ef [llvm-objcopy] Add support for large indexes
Because of -ffunction-sections (and maybe other use cases I'm not aware of?) it
can occur that we need more than 0xfeff sections but ELF dosn't support that
many sections. To solve this problem SHN_XINDEX exists and with it come a whole
host of changes for section indexes everywhere. This change adds support for
those cases which should allow llvm-objcopy to copy binaries that have an
arbitrary number of sections.

Differential Revision: https://reviews.llvm.org/D42516

llvm-svn: 326940
2018-03-07 19:59:15 +00:00
Alexander Richardson
6c85992c6d [llvm-objcopy] Use the full filename in --add-gnu-debuglink
Summary:
The current implementation was writing the file name without the extension
whereas GNU objcopy writes the full filename. With this change GDB will now
load the .debug file instead of silently ignoring it.

Reviewers: jakehehrlich, jhenderson

Reviewed By: jakehehrlich

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D43474

llvm-svn: 325528
2018-02-19 19:53:44 +00:00
Jake Ehrlich
6452b11fd8 [llvm-objcopy] Fix handling of zero-size segments in llvm-objcopy
Some ELF files produced by lld may have zero-size segment placeholders as shown
below. Since GNU_STACK Offset is 0, the current code makes it the lowest used
offset, and relocates all the segments over the ELF header. The resulting
binary is total garbage.

This change fixes how llvm-objcopy handles PT_PHDR properlly by treating ELF
headers and the program header table as segments to allow the layout algorithm
decide where those should go.

Author: vit9696

Differential Revision: https://reviews.llvm.org/D42872

llvm-svn: 325189
2018-02-14 23:31:33 +00:00
Jake Ehrlich
9634e18ffe Reland "[llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects"
Somehow I reverted changes I made in a previous Reland. This change re-relands
unconfusing a varible name with a type name.

llvm-svn: 323494
2018-01-26 02:01:37 +00:00
Jake Ehrlich
41d9d7b16f Reland "[llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects"
I had more unused varibles. This change removes those to get rid of warnings.

llvm-svn: 323493
2018-01-26 01:48:12 +00:00
Jake Ehrlich
c0e9bee781 Reland "[llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects"
Added line to output the proper files in the output to binary case.

llvm-svn: 323489
2018-01-26 01:17:35 +00:00
Jake Ehrlich
82d61211b2 Revert "Reland "[llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects""
Tests were working on my system because the old correct files were left over
and the new bug was that the output files were not being output at all.
Consequently the test work on my system but fail on any other system.

This reverts commit r323484.

llvm-svn: 323486
2018-01-26 00:38:30 +00:00
Jake Ehrlich
6d88ffd256 Reland "[llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects"
I named a varible the same as a type which caused a warning. I also had unamed varibles.

llvm-svn: 323484
2018-01-26 00:19:30 +00:00
Jake Ehrlich
76e9110f3d [llvm-objcopy] Refactor llvm-objcopy to use reader and writer objects
While writing code for input and output formats in llvm-objcopy it became
apparent that there was a code health problem. This change attempts to solve
that problem by refactoring the code to use Reader and Writer objects that can
read in different objects in different formats, convert them to a single shared
internal representation, and then write them to any other representation.

New classes:
Reader: the base class used to construct instances of the internal
representation
Writer: the base class used to write out instances of the internal
representation
ELFBuilder: a helper class for ELFWriter that takes an ELFFile and converts it
to a Object
SectionVisitor: it became necessary to remove writeSection from SectionBase
because, under the new Reader/Writer scheme, it's possible to convert between
ELF Types such as ELF32LE and ELF32BE. This isn't possible with writeSection
because it (dynamically) depends on the underlying section type *and*
(statically) depends on the ELF type. Bad things would happen if the underlying
sections for ELF32LE were used for writing to ELF64BE. To avoid this code smell
(which would have compiled, run, and output some nonsesnse) I decoupled writing
of sections from a class.
SectionWriter: This is just the ELFT templated implementation of
SectionVisitor. Many classes now have this class as a friend so that the
writing methods in this class can write out private data.
ELFWriter: This is the Writer that outputs to ELF
BinaryWriter: This is the Writer that outputs to Binary
ElfType: Because the ELF Type is not a part of the Object anymore we need a way
to construct the correct default Writer based on properties of the Reader. This
enum just keeps track of the ELF type of the input so it can be used as the
default output type as well.

Object has correspondingly undergone some serious changes as well. It now has
more generic methods for building and manipulating ELF binaries. This interface
makes ELFBuilder easy enough to use and will make the BinaryReader/Builder easy
to create as well. Most changes in this diff are cosmetic and deal with the
fact that a method has been moved from one class to another or a change from a
pointer to a reference. Almost no changes should result in a functional
difference (this is after all a refactor). One minor functional change was made
and the result can be seen in remove-shstrtab-error.test. The fact that it
fails hasn't changed but the error message has changed because that failure is
detected at a later point in the code now (because WriteSectionHeaders is a
property of the ElfWriter *not* a property of the Object). I'd say roughly
80-90% of this code is cosmetically different, 10-19% is different but
functionally the same, and 1-5% is functionally different despite not causing a
change in tests.

Differential Revision: https://reviews.llvm.org/D42222

llvm-svn: 323480
2018-01-25 22:46:17 +00:00
Jake Ehrlich
ea07d3cf65 [llvm-objcopy] Add --add-gnu-debuglink
This change adds support for --add-gnu-debuglink to llvm-objcopy

Differential Revision: https://reviews.llvm.org/D41731

llvm-svn: 323477
2018-01-25 22:15:14 +00:00
Aaron Ballman
4af8836398 Revert r322132; it appears to be an accidental commit, based on the commit message. The original author of the commit has not commented on whether this was accidental or purposeful, so if this revert is in error, the author can re-commit with an actual commit message.
llvm-svn: 323466
2018-01-25 21:08:23 +00:00
Aaron Ballman
09f46a76d9 Reverting r323463 as it appears to be an accidental commit. Regardless, it broke a lot of build bots, so reverting back to green.
http://lab.llvm.org:8011/builders/lldb-amd64-ninja-netbsd8/builds/9294
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/24084
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-lnt/builds/9567

llvm-svn: 323465
2018-01-25 21:03:38 +00:00
Jake Ehrlich
df35594077 tmp
llvm-svn: 323463
2018-01-25 20:24:17 +00:00
Jake Ehrlich
46814bee63 [llvm-objcopy] Use physical instead of virtual address when aligning and placing sections in binary
For sections with different virtual and physical addresses, alignment and
placement in the output binary should be based on the physical address.

Ran into this problem with a bare metal ARM project where llvm-objcopy added a
lot of zero-padding before the .data section that had differing addresses. GNU
objcopy did not add the padding, and after this fix, neither does llvm-objcopy.

Update a test case so a section has different physical and virtual addresses.

Fixes B35708

Authored By: Owen Shaw (owenpshaw)

Differential Revision: https://reviews.llvm.org/D41619

llvm-svn: 323144
2018-01-22 19:27:30 +00:00
Jake Ehrlich
99482fda95 temp
llvm-svn: 322132
2018-01-09 23:00:25 +00:00
Jake Ehrlich
27a29b0290 [llvm-objcopy] Add --localize-hidden option
This change adds support in llvm-objcopy for GNU objcopy's --localize-hidden
option. This option changes every hidden or internal symbol into a local symbol.

llvm-svn: 321884
2018-01-05 19:19:09 +00:00
Jake Ehrlich
30d927a128 [llvm-objcopy] Add support for visibility
I have no clue how this was missed when symbol table support was added. This
change ensures that the visibility of symbols is preserved by default.

llvm-svn: 321681
2018-01-02 23:01:24 +00:00
Jake Ehrlich
e8437de727 [llvm-objcopy] Add option to add a progbits section from a file
This change adds support for adding progbits sections with contents from a file

Differential Revision: https://reviews.llvm.org/D41212

llvm-svn: 321047
2017-12-19 00:47:30 +00:00
Jake Ehrlich
777fb00a76 [llvm-objcopy] Reformat everything using clang-format -i
Overtime some non-clang formatted code has creeped into llvm-objcopy. This
patch fixes all of that.

Differential Revision: https://reviews.llvm.org/D41262

llvm-svn: 320856
2017-12-15 20:17:55 +00:00
Jake Ehrlich
d49c92b124 [llvm-objcopy] Change -O binary to respect section removal and behave like GNU objcopy
The original -O binary implementation just copied segment data from the
object and dumped it into a file. This doesn't take into account any
operations performed on objects such as section removal. GNU objcopy has
some specific behavior that we'd also like to respect. For instance
using -O binary and -j <some_section> will dump <some_section> to a
file. This change implements GNU objcopy style -O binary to as close of
an approximation as I can determine.

Differential Revision: https://reviews.llvm.org/D39713

llvm-svn: 318324
2017-11-15 19:13:31 +00:00
Jake Ehrlich
13153eef56 [llvm-objcopy] Fix bug in how segment alignment was being handled
Just aligning segment offsets to segment alignment is incorrect and also
wastes more space than is needed. The requirement is that p_offset ==
p_addr modulo p_align *not* that p_offset == 0 modulo p_align. Generally
speaking we've been using p_addr == 0 modulo p_align. In fact yaml2obj
can't even produce a valid situation which causes llvm-objcopy to
produce incorrect results because alignment and offset were both
inherited from the sections the program header covers. This change fixes
this bad behavior in llvm-objcopy.

Differential Revision: https://reviews.llvm.org/D39132

llvm-svn: 317284
2017-11-02 23:24:04 +00:00
Eugene Zelenko
0ad18f888e [dsymutil, llvm-objcopy] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 317123
2017-11-01 21:16:06 +00:00
Zachary Turner
41a9ee98f9 Revert "[ADT] Make Twine's copy constructor private."
This reverts commit 4e4ee1c507e2707bb3c208e1e1b6551c3015cbf5.

This is failing due to some code that isn't built on MSVC
so I didn't catch.  Not immediately obvious how to fix this
at first glance, so I'm reverting for now.

llvm-svn: 315536
2017-10-11 23:54:34 +00:00
Zachary Turner
337462b365 [ADT] Make Twine's copy constructor private.
There's a lot of misuse of Twine scattered around LLVM.  This
ranges in severity from benign (returning a Twine from a function
by value that is just a string literal) to pretty sketchy (storing
a Twine by value in a class).  While there are some uses for
copying Twines, most of the very compelling ones are confined
to the Twine class implementation itself, and other uses are
either dubious or easily worked around.

This patch makes Twine's copy constructor private, and fixes up
all callsites.

Differential Revision: https://reviews.llvm.org/D38767

llvm-svn: 315530
2017-10-11 23:33:06 +00:00
Jake Ehrlich
f03384dce7 Reland "[llvm-objcopy] Add support for --strip-sections to remove all section headers leaving only program headers and loadable segment data"
ubsan caught an issue I made where I was converting a null pointer to a
reference.

elf utils implements a particularly extreme form of stripping that I'd
like to support. eu-strip has an option called "strip-sections" that
removes all section headers and leaves only program headers and the
segment data. I have implemented this option partly as a test but mainly
because in Fuchsia we would like to use this option to minimize the size
of our executables. The other strip options that are on my list include
--strip-all and --strip-debug. This is a preliminary implementation that
I'd like to start using in Fuchsia builds if possible. This change
implements such a stripping option for llvm-objcopy

Differential Revision: https://reviews.llvm.org/D38335

llvm-svn: 315484
2017-10-11 18:09:18 +00:00
Jake Ehrlich
d9a283463a Revert "[llvm-objcopy] Add support for --strip-sections to remove all section headers leaving only program headers and loadable segment data"
This reverts commit rL315412

llvm-svn: 315417
2017-10-11 02:42:29 +00:00