Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow 'ninja -t compdb' to accept targets #2497

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

JamesWidman
Copy link

This fixes #1544.

Here we use the same commit as in #2319, plus one more commit so that the option is spelled --targets.

Many thanks to @lk-chen for the original work in #1546, and to @csmoe for carrying it forward in #2319!

@jhasse jhasse added this to the 1.13.0 milestone Sep 20, 2024
@mcprat
Copy link
Contributor

mcprat commented Sep 20, 2024

squash to 1 commit to get rid of the merge commits and combine the original with your rename, add yourself to the tags at the end of the original commit

it would be nice if we can find a way to get rid of stringstream (for lower binary size?)

I still think the remaining arguments should not require a long option at all or otherwise -- if its not possible to avoid long opts. Isn't it safe to assume everything after -t compdb will be a target? if not then whatever else can be after that should be the type of argument with it's own unique long option

@JamesWidman
Copy link
Author

@mcprat Agreed on squashing/rebasing, and i will look into the stringstream thing.

as for the command-line syntax: i wrote about this in #2319 (though i forgot to tag you there; sorry!), but it would be good to have it inline here too:

You wrote:

Would it be better to shift out arguments so that no flag is required for the targets list?

so instead of

-t compdb --target foo,bar

you would have

-t compdb foo bar

Prior to the commits in this PR, the output of ninja -t compdb -h is:

usage: ninja -t compdb [options] [rules]

options:
  -x     expand @rspfile style response file invocations

so ninja -t compdb foo bar already means "emit every compile command that uses rule foo or rule bar", and we probably don't want to break that.

or if a flag is really necessary to break out of flag-only parsing, use syntax like in git

-t compdb -- foo bar

so they can be space separated instead of comma separated

but that means the syntax would become:

ninja -t compdb [options] [rules] -- [targets]

...which means that users will need to remember that -t compdb foo -- bar means "rule foo and target bar", as opposed to "target foo and rule bar".

How would you feel about a short option instead of a long option? (meaning: -t TARGETS instead of --targets TARGETS)

Also: if we go with an option, are you ok with making the target names comma-delimited? (i think the main reason to use commas is so that we stick to the format expected by getopt()/getopt_long(), where the set of all targets would be returned via optarg.)

Fwiw, in bash 4.2 and later, you can join an array with commas using the printf builtin like so:

echo $(printf "%s," ${targets[*]})

@JamesWidman
Copy link
Author

JamesWidman commented Sep 20, 2024

@mcprat oh wait, when you wrote:

if not then whatever else can be after that should be the type of argument with it's own unique long option

are you proposing something like:

ninja -t compdb --target t1 --target t2 --target tN rule1 rule2 ruleN

...or maybe:

ninja -t compdb -t t1 -t t2 -t tN rule1 rule2 ruleN

...?

@JamesWidman
Copy link
Author

@mcprat btw, i assumed that we don't want to break the pre-existing [rules] interface because i assume that someone, somewhere, depends on that in a script or something, but personally i can't see much of a use for it if we can specify targets.

Do we want to make the breaking change of removing the [rules] syntax (or replacing it with an option that allows something like -r rule1 -r rule2 [...])?

@JamesWidman
Copy link
Author

JamesWidman commented Sep 21, 2024

@mcprat i just thought of another way we could do this:

up until now, i've been stuck thinking that --targets is an option that must have an argument, but we could alternatively tell getopt()/getopt_long() that --targets (or -t) doesn't take an argument. Then we could change the form of the command, in a non-breaking way, to this:

ninja -t compdb [options] [items]

...where the [items] are interpreted as rules unless one of the [options] is -t/--targets, in which case the [items] are interpreted as targets. So in effect, we would have two alternative forms:

ninja -t compdb --targets [other_options]  target1 target2 [...] targetN
ninja -t compdb [other_options] rule1 rule2 [...] ruleN

...which, although it would add an option, it would at least mean that targets are separated by spaces as you requested.

(In this case, i would have a weak preference for the long form (--targets rather than -t), just because it follows another -t option that means something completely different (the "tool" option). It's not a deal-breaker for me if we go with the single-letter spelling, though.)

@mcprat
Copy link
Contributor

mcprat commented Sep 21, 2024

@JamesWidman I'm aware of your replies in the other PR. I was reiterating, but also clarifying that neither a short or long option would be the ideal (closer to POSIX-like behavior), because it seems like what I said previously was interpreted as only preferring short option or -- over long option.

All of this definitely clears things up, it is more complicated than I thought...

Here's an idea... what's stopping the parsing stage from checking both the list of targets and list of rules for each argument? Without having looked into it, there's probably different functions used depending on whether the argument is a rule or a target. Maybe each argument can be passed to both, a failure can be caught, and if only one has a problem that's expected, but both failing (or even if neither are failing) would indicate an actual problem.

@mcprat
Copy link
Contributor

mcprat commented Sep 21, 2024

or rather, since this is a printing-only tool, it can just ignore any errors...

@JamesWidman
Copy link
Author

JamesWidman commented Sep 21, 2024

I was reiterating, but also clarifying that neither a short or long option would be the ideal (closer to POSIX-like behavior),

@mcprat I am trying to understand:

  1. why is it desirable to avoid the introduction of a new option (like --targets or -t)?
  2. how does POSIX enter into it?

(I do plan on getting to your other points, but first, i really want to understand the answers to these questions.)

@mcprat
Copy link
Contributor

mcprat commented Sep 21, 2024

  1. why is it desirable to avoid the introduction of a new option (like --targets or -t)?

it's not really an option but rather more input, as it doesn't adjust existing outputs but rather just adds to it. consider the way the manual is written right now

given a list of rules, each of which is expected to be a C family language compiler rule whose first input is the name of the source file, prints on standard output a compilation database...

instead of now having to describe that there's different ways that one adds a rule to the list vs how one adds a target to the list, it would be nice if the manual can just say:

given a list of rules and/or targets...

  1. how does POSIX enter into it?

I consider the design of make as a general guide. even though most consider it a product of GNU, it is in fact defined by POSIX as well, and they try to mostly stay conformant. even though Ninja may not explicitly claim to be POSIX conforming we ought to follow it when possible.

POSIX has a section called "Utility Syntax Guidelines" that's used to form the standard syntax for make. there is mention there about the usage of -- and how the order of arguments can affect the output, and that there should be only one argument to each option. I don't see much about how the definition of an option differs from the definition of a normal operand however...

@JamesWidman
Copy link
Author

JamesWidman commented Sep 23, 2024

@mcprat First: i think my confusion might be shrinking; thank you!

i didn't quite understand what you meant here:

  1. why is it desirable to avoid the introduction of a new option (like --targets or -t)?

it's not really an option but rather more input, as it doesn't adjust existing outputs but rather just adds to it.

...but i think i understand this part:

consider the way the manual is written right now

given a list of rules, each of which is expected to be a C family language compiler rule whose first input is the name of the source file, prints on standard output a compilation database...

instead of now having to describe that there's different ways that one adds a rule to the list vs how one adds a target to the list, it would be nice if the manual can just say:

given a list of rules and/or targets...

so... if i understand you correctly, it sounds like you're saying something like:
"The user interface for producing a compilation database for a given set of ninja rules ought to be consistent with the user interface for producing a compilation database for a given set of ninja targets."

Is that about right?

(If so: i see that the consistency that you're asking for could make it easier for a user to transition from using rules to using targets or vice-versa, and that would end the confusion that prompted my first question above.)

Second:

POSIX has a section called "Utility Syntax Guidelines"

i started reading Issue 8 (IEEE Std 1003.1™-2024 Edition) as well as a draft from 1991, and i just realized that you might be targeting some other version in between (one that is old enough that it is implemented by all the platforms the ninja project wants to run on, but new enough that it specifies all the OS features that the project wants ninja to depend on).

i checked CONTRIBUTING.md, the wiki, and the ninja manual, but i didn't see any mention of POSIX.

so: what version of POSIX should we be reading/targeting?

Also: would it make sense to add a mention of that version of POSIX to the wiki and/or CONTRIBUTING.md to mention that version? (I can see how POSIX compliance is another way to support consistency of user interfaces (in this case, consistency of syntax between ninja and POSIX utilities), and also helps us avoid breaking the rule-of-least-surprise. Thank you for the pointer!)

src/ninja.cc Outdated
@@ -945,6 +948,32 @@ void printCompdb(const char* const directory, const Edge* const edge,
printf("\"\n }");
}

struct DependentEdgeCollector {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this class definition to src/graph.h and provide proper documentation + unit-tests for it. See InputsCollector there for inspiration.

I also recommend to make depend_edges a member of the class, and add a method like TakeResult() to move the final result out of the instance one all root targets have been visited.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this class definition to src/graph.h

is it ok if i move it to a new header file instead? It's very compdb-specific, and only used by ninja.cc and graph_test.cc, so it would be nice if changing it didn't cause a lot of unrelated source files to be recompiled

src/ninja.cc Outdated
std::vector<Edge*> user_interested_edges;
DependentEdgeCollector deps_edge_collector;

while (std::getline(iss, target, ',')) {
Copy link
Contributor

@digit-google digit-google Sep 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use SplitStringPiece() from string_piece_util.h instead for simpler and smaller generated code. Use std:: prefixes for all standard types, as we are trying to move away from using namespace std. E.g.:

for (StringPiece target_piece : SplitStringPiece(optarg, ',')) {
  std::string err;
  std::string target = target_piece.AsString();
  Node* node = CollectTarget(target.c_str(), &err);
  ...
}

src/ninja.cc Outdated
return 1;
}
if (!deps_edge_collector.CollectFrom(node, &user_interested_edges)) {
return 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silent failure without a message explaning what's going on is a bad user experience :-/
Besides, it looks like your CollectFrom() implementation cannot return false, so get rid of the return value and the condition check here.

src/ninja.cc Outdated
@@ -945,6 +948,32 @@ void printCompdb(const char* const directory, const Edge* const edge,
printf("\"\n }");
}

struct DependentEdgeCollector {
bool CollectFrom(Node* node, std::vector<Edge*>* depend_edges) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: use const Node* for the first argument :-)

@digit-google
Copy link
Contributor

We do really not care about following POSIX guidelines or imitating the GNU Make design.
On the other hand we do care about developer workflows not breaking when introducing changes.

As such, changing how non-option arguments to -t compdb are interpreted is a strict no-go.

It is ok to introduce new options, even if they change the tool's behavior, as long as everything is properly documented, and there is no place for ambiguity. If this is not possible, or makes the UI too cluncky, introducing a new tool might be the better option.

@digit-google
Copy link
Contributor

In this specific case, using --targets as a separator between rules and targets seems ok, and no one likes commas on the command-line. What I mean is something like -t compdb [-x] [rule]... --targets [target]..., but I am not sure that getopt supports this here.

@JamesWidman
Copy link
Author

JamesWidman commented Sep 23, 2024

We do really not care about following POSIX guidelines or imitating the GNU Make design.

@digit-google Works for me, but is there consensus among maintainers on this point? (I got the sense from @mcprat that some POSIX-ness and Make-ness was desired. I don't feel a strong preference either way, but it would help to have unambiguous direction.)

On the other hand we do care about developer workflows not breaking when introducing changes.

[thumbs up]

As such, changing how non-option arguments to -t compdb are interpreted is a strict no-go.

So for example (and please correct me if i'm wrong):

  1. in ninja -t compdb item1 item2, we will continue to interpret the two items as rules only (and not as targets);
  2. likewise, we will NOT introduce a form like ninja -t compdb --targets item1 item2 where the two items are interpreted as targets

[edit: ...or would item (2) here be viable because it uses a new option that changes the tool's behavior?]

It is ok to introduce new options, even if they change the tool's behavior, as long as everything is properly documented, and there is no place for ambiguity.

Sounds good to me!

If this is not possible, or makes the UI too cluncky, introducing a new tool might be the better option.

... and by "tool" you mean a new sub-command, right? For example, maybe like:

ninja -t compdb-targets [options] [targets...]

@digit-google
Copy link
Contributor

Yes, except for 2, since -t comptb --targets item1 item2 would not break existing workflows, since --targets was not supported before this change.

Personally, I do not care about sticking to Posix guidelines. Ninja is supposed to be cross-platform anyway, and there are plenty of command-line parsers (like python argparse or Golang's flags) which do not follow them, and end up perfectly ok (or even great in the case of argparse, imho). I prefer ensuring that command-line options are simple to understand, provide correct and reproducible outputs, and are well documented in --help and other docs (Ninja had issues with all these in the past, but the situation is slowly improving).

I also forgot to ask for a regression test for the tool in output_test.py to be added. One that experiments all modes (rules, targets, and rules + targets) preferrably. This is also important to avoid breaking stuff unexpectedly.

@JamesWidman
Copy link
Author

JamesWidman commented Sep 26, 2024

@digit-google btw, what's our C++ version for compiling ninja? (i grepped for CXX_STANDARD in the cmake files but found nothing.)

i see uses of auto and range-based for loops, so that means we require at least C++11.

i ask because i wanted to use a structured binding in a place where i thought it would help to add clarity, but that's a C++17 feature, which would require setting CXX_STANDARD and CXX_STANDARD_REQUIRED.

Note, according to the clang website:

By default, Clang 16 or later builds C++ code according to the C++17 standard.

...and likewise, GCC 11 defaults to C++17.

So if we're not going to depend on C++14 or C++17 features, we may want to set CXX_STANDARD to 11 so that we get the desired compile errors in the event where people unwittingly uses more recent features.

@digit-google
Copy link
Contributor

IIRC, the goal is to have the core Ninja sources building with C++11 support only, in order to allow building the tool on older systems (in particular with ./configure.py --bootstrap rather than CMake).

However, the tests now rely on GTest which mandates C++14 though (and soon C++17), which is why CXX_STANDARD is not applied to them in our CMakeList.txt file.

I think these requirements were in the top-level CONTRIBUTING.md file, but no longer are there.

I wouldn't mind switching to C++17 personally, but @jhasse is the authority on this specific issue (and this may require updating some of our CI recipes).

@JamesWidman
Copy link
Author

JamesWidman commented Sep 27, 2024

@digit-google @jhasse this is a little off-topic for this PR, but: what's our policy on GNU core-language extensions)?

some pros/cons (in case this hasn't come up before):

pros:

con:

  • Allowing GNU extensions would rule out the use of MSVC to compile ninja. (i don't know whether that would be a deal-breaker for the ninja project, but given that speed is one of the top priorities, i can see why you'd want to keep MSVC compatibility in case the code that MSVC generates is ever faster for use-cases that ninja users care about).

@digit-google
Copy link
Contributor

I believe right now we absolutely want to build with the MSVC toolchain, and support non-GNU and non-Clang toolchains, so these are a deal breakers.

@JamesWidman JamesWidman marked this pull request as draft September 27, 2024 13:41
@JamesWidman
Copy link
Author

JamesWidman commented Sep 27, 2024

@digit-google

In this specific case, using --targets as a separator between rules and targets seems ok, and no one likes commas on the command-line. What I mean is something like -t compdb [-x] [rule]... --targets [target]..., but I am not sure that getopt supports this here.

so it turns out that this is doable, but it requires using getopt_long() in a way that is different from the normal usage pattern for getopt()/getopt_long(). To simplify things for NinjaMain::ToolCompilationDatabase(), it seems best to move argument parsing for compdb into a separate routine that (1) parses all arguments and (2) returns a structure that's easier to act on.

at the moment i'm working on regression tests and documentation, but here's a preview of the argument parsing routine (for early feedback on the general shape of this approach):

(Before my next push, i will review the coding style guidelines, so hopefully that push will not contain e.g. any naming style violations that might appear below.)

Compdb parseCompdbArguments(int argc, char* argv[]) {
  Compdb ret;
  //
  // grammar:
  //     ninja -t compdb [-hx] [rules] [--targets targets]
  //
  // in phase 1 of argument parsing, we process [-hx] and "--targets". The
  // pseudo-option "--targets" is conceptually more like "--" than an option.
  // Traditionally, "--" means "every argument that follows is an operand [i.e.,
  // a non-option argument]". Here, "--targets" means "every argument that
  // follows is grammatically an operand, and semantically a target. Also, rule
  // operands may precede this argument."
  //
  // If getopt_long returns an instance of "--targets", we store its `optind`
  // (see `man 3 getopt`) as the starting index for the list of targets, and we
  // store the previous `optind` as the starting index for the list of rules.
  //
  // in phase 2 of argument parsing, we process the list of rules (if any) and
  // the list of targets (if any).

  /* [...] */

  return ret;
}

...where Compdb is:

struct Compdb {
  enum class Action {
    display_help_and_exit,
    emit_all_commands,
    emit_userSpecified_commands
  };

  Action action;
  EvaluateCommandMode eval_mode = ECM_NORMAL;

  typedef std::vector<StringPiece> ArgList;
  ArgList targets;
  ArgList rules;
};

...and the usage looks like this:

int NinjaMain::ToolCompilationDatabase(const Options* options, int argc,
                                       char* argv[]) {
  Compdb compdb = parseCompdbArguments(argc, argv);

  switch (compdb.action) {
  /* [...] */
  }

  return 0;
}

Feedback welcome!

(Note: the comment about "storing optind" above is only an implementation detail of parseCompdbArguments(); it is not visible to the caller, who will not have to think about optind, getopt(), or argc/argv.)

@mcprat
Copy link
Contributor

mcprat commented Sep 28, 2024

POSIX has a section called "Utility Syntax Guidelines"

i started reading Issue 8 (IEEE Std 1003.1™-2024 Edition)...

I did originally read the section from Issue 7, but I don't see anything different from Issue 8
It's base definitions section 12.2
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html

We do really not care about following POSIX guidelines or imitating the GNU Make design.

Works for me, but is there consensus among maintainers on this point? (I got the sense from @mcprat that some POSIX-ness and Make-ness was desired.

I'm not bringing up POSIX-ness or Make functionality to make things harder for us, it should actually make things easier. I understand Ninja does not have POSIX-compliance as a major goal, but I believe it should be done whenever it is easy to do so. POSIX likely recommends against an option having multiple arguments because it's simply not efficient, and doesn't match what most consider to be an "option" compared to what one would define as "input". I personally don't see why we need the secondary stage of getopt processing or similar alternatives at all...

imagine instead of

ninja -t compdb [-x] [RULES...] [--targets TARGETS...]

or

ninja -t compdb [-x] [--targets TARGETS...] [RULES...]

we can just have

ninja -t compdb [-x] [TARGETS...] [RULES...]
or just writing it the other way around...
ninja -t compdb [-x] [RULES...] [TARGETS...]

where the order of operands affects the order of output, but not the actual total content (amount of JSON elements).

Then in the manual you don't have to explain a new "option", rather it can simply read:
"given a list of rules and/or targets ... prints on standard output a compilation database ..."

I'm going to try to get that method to work myself, to make sure I'm not preaching for something that isn't possible to begin with... I wanted to have it done already but I've been busy...

@jhasse jhasse marked this pull request as ready for review September 28, 2024 07:25
@jhasse
Copy link
Collaborator

jhasse commented Sep 28, 2024

Sry misclicked.

Regarding C++ standard: I'm okay with everything that works relativly easy on the oldest supported RHEL version (8 currently).

I think it would be best to stick with a command line way that would also work easily with other command line parsers.

@digit-google
Copy link
Contributor

@mcprat , what you propose is ambiguous, and I assure you that someone, somewhere is going to have a rule and a target with the same name, and won´t understand why they cannot use the tool as expected. There needs to be a explicit separation in the parsing format. Or simply introduce a new tool name for targets, nobody will ever get confused.

@digit-google
Copy link
Contributor

Regarding C++, it looks like we should be able to switch to C++17 (to be performed in a separate CL of course).

this page says RHEL ships with GCC 8.x or 9.x, and that page says that all C++17 features are supported starting with GCC 8 (and even GCC 7 if you don't care about full support of template deductions).

Similarly, this page shows that VS 2019, which we currently use in CI, supports C++17 (unlike VS 2017 which still lacks some features).

@mcprat
Copy link
Contributor

mcprat commented Sep 28, 2024

a rule and a target with the same name

yes, this is a real problem. but options without arguments like --targets or --rules can block the printing of one type or the other.

if that's not acceptable, then I would vote for separate tools with separate names.

@JamesWidman JamesWidman marked this pull request as draft September 28, 2024 13:56
@JamesWidman
Copy link
Author

POSIX likely recommends against an option having multiple arguments because it's simply not efficient,

it turns out that, as soon as getopt_long() returns 't' to indicate that it recognized the "--targets" option, we can stash the relevant optind values and break out of the loop over calls to getopt_long(). So, in my current implementation, "--targets" is just like "--" in the sense that it immediately terminates the processing of options by getopt_long(). That seems sufficiently efficient...?

@JamesWidman
Copy link
Author

@mcprat One reason why i prefer having some explicit option (or separate sub-tool, as in ninja -t compdb_targets) is that i tend to use these tools in scripts, and i want those scripts to be as readable as possible so that it's as easy as possible for the reader (possibly my distant-future self) to (re-)build a good understanding of what a particular invocation does & why.

as a user reading a shell script, compdb --targets foo tells me unambiguously that "foo" is intended/expected to be a target. And i consider this desirable, because in general, i like to avoid being confused about the meaning of the code that i read (including shell commands).

But also, "--targets" makes it clear to ninja that subsequent arguments must be taken as names of targets, and that makes the diagnosis of errors very very simple & straightforward.

@JamesWidman
Copy link
Author

Sry misclicked.

no worries (:

I think it would be best to stick with a command line way that would also work easily with other command line parsers.

Sorry; i'm not sure i understand what you mean here. What do you mean by "other command line parsers", and how would they be affected by the suggested [rules] [--targets targets] syntax?

@JamesWidman
Copy link
Author

@mcprat , what you propose is ambiguous, and I assure you that someone, somewhere is going to have a rule and a target with the same name, and won´t understand why they cannot use the tool as expected.

Agreed; Hyrum's Law definitely applies here.

There needs to be a explicit separation in the parsing format. Or simply introduce a new tool name for targets, nobody will ever get confused.

I'm happy to implement ninja -t compdb_targets [targets] as an alternative to ninja -t compdb --targets if that's what people want.

@digit-google how should we/i proceed?

(Note, -t compdb_targets would seem to imply that you cannot mix rules & targets in one command and get a de-duplicated set of results (which is fine by me, at the moment, since i don't see a use-case for it).)

todo:

 - add more unit tests
 - review coding-style guidelines & fix any violations
 - update manual to mention the new syntax:
     ninja -t compdb [-hx] [rules] [--targets targets]

Co-authored-by: Linkun Chen <[email protected]>
Co-authored-by: csmoe <[email protected]>
Co-authored-by: James Widman <[email protected]>
@JamesWidman
Copy link
Author

the commit in this force-push is obviously very much a work-in-progress, but it seems like people might benefit from experimenting with its proposed command syntax:

ninja -t compdb [-hx] [rules] [--targets targets]

(note, if you don't use "--targets", then all non-option arguments after compdb are interpreted as rules, as is the status quo.)

At this point, it seems like the main open question is whether we want this --targets command-line interface as opposed to a new tool, whose grammar would probably look something like this:

ninja -t compdb-targets [-hx] [targets]

(the compdb-targets tool is currently not implemented, but i can do that next if anyone wants to experiment with it. At this point, it would be very easy for me to add.)

i guess the main question is: does anyone actually have a use-case for merging the commands from a mix of both rules and targets (as implemented in the most recent push)? If not, then maybe it's better to proceed with -t compdb-targets instead of -t compdb --targets

@digit-google thoughts?

@JamesWidman
Copy link
Author

coding note: since this represents an overhaul of -t compdb rather than a minor change, i moved all the compdb stuff in ninja.cc to just after the definition of NinjaMain::ToolRestat() so as to reduce diff noise.

@JamesWidman
Copy link
Author

JamesWidman commented Sep 30, 2024

update: i was just playing with jq, and it turns out that it can be used to merge & de-duplicate objects from two different json files. So, if you want to have a single compile_commands.json that contains commands for both a specified set of rules and a specified targets (for some reason), with duplicates removed, you could do:

ninja -t compdb rule1 rule2 [...] > a.json
ninja -t compdb --targets target1 target2 [...] > b.json
echo '[]' | jq --slurpfile a a.json --slurpfile b b.json '($a[] + $b[]) | unique_by(.output)' > compile_commands.json

(the jq function unique_by() sorts the output, but there are ways to prevent that.)

Since this merging-and-deduplication can be done with jq, i propose that this PR should discontinue pursuit of the --targets option and instead proceed with the implementation of a new tool, which will look something like this:

ninja -t compdb-targets [-hx] [targets]

@digit-google @jhasse What do you think?

@eli-schwartz
Copy link

I wouldn't mind switching to C++17 personally, but @jhasse is the authority on this specific issue (and this may require updating some of our CI recipes).

I continue to be regularly confused every time I look at ninja issues:

Are you, @digit-google, a maintainer authority on ninja or are you not? Sometimes you say "I am but a humble contributor", most of the time you use the possessive "our", sometimes you use wording such as "I require you to do X. Note that jhasse is the authority on Y".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Allow 'ninja -t compdb' accept one target
5 participants