No Submodule Installation

[mateusz-]

Hello,

The installation directions state that:

After installing the requirements, continue by cloning the Embedded Proto repo. We advised using Embedded Proto as a submodule in your project. This way, you can track the version of Embedded Proto with the version of your project.

Let's say I would like to avoid using a submodule and embed EmbeddedProto within my repository (i.e., remove EmbeddedProto's .git folder and commit all files). This is what happens when I do that:

$ git clone [email protected]:Embedded-AMS/EmbeddedProto.git EmbeddedProto
Cloning into 'EmbeddedProto'...
...
Resolving deltas: 100% (3154/3154), done.

$ cd EmbeddedProto
$ rm -rf .git
$ python setup.py
================================================================================
|                                                                              |
| Three simple things you can do to help improve Embedded Proto:               |
|  * Give private feedback:                                                    |
|        https://EmbeddedProto.com/feedback                                    |
|  * Report an issue in public on Github:                                      |
|        https://github.com/Embedded-AMS/EmbeddedProto/issues                  |
|  * Stay up to date on Embedded Proto via our User mailing list:              |
|        https://EmbeddedProto.com/signup                                      |
|                                                                              |
================================================================================

Checking your Protoc version [Success]
Creating a virtual environment for Embedded Proto. [Success]
Installing EmbeddedProto in the virtual environment. [Fail]
  ERROR: Command errored out with exit status 1:
...
raise LookupError(
  LookupError: setuptools-scm was unable to detect version for ...\EmbeddedProto.
  Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
  For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
  ----------------------------------------
...

The error is clear enough, if I add back the .git, everything works fine. I truncated the output above.

My company purchased a commercial license.

[BartHertog]

Hi @mateusz,

Good that you write in. You are not the only one that desires a similar feature. People have also asked for a global installation.

Here is a little behind-the-scenes:
Currently, v4 of Embedded Proto is in development. That version will likely support installation with pip from PyPI without cloning the repo. Usability should increase with this release but there are still items to be resolved. A small example Embedded Proto comprises of two parts:

1. One is the Python code and templates for the plugin.
2. The second part is the code included for the embedded build.
   The installation from PyPI will install the first and possibly install the second. But to include it in the firmware build you need to know where the files from the second item are located. Python packages are not built to distribute C++ code. A regular Linux package could solve that but not everyone is using Linux so the solution should also work on Windows.

Back to now:
To help you out I will look into something like a workaround until v4 is released. I am thinking of a command line option to the setup script to do a global installation. Then you would only have to include the files from the src folder in your repo. There are two drawbacks:

1. Every developer and the CI server should install Embedded Proto that way, and
2. you should make sure that the versions of the source code and installations stay in sync.

I will let you know how this progresses.

Best regards,

Bart

[mateusz]

Mmmm protobufs for embedded 😋

[BartHertog]

The simple solution was not going to work. Currently, the development branch of version 4 is the shortest route to what you like to use. It does need more work though. I'll keep you updated on the progress.

[BartHertog]

Oke here is something you can try.

Before you try this please understand, that this is still in the development stage, not all is complete yet but the general direction and idea should be clear. I would not suggest you use this in active development right away.

What is new with version 4:

• You can install Embedded Proto from PyPi the Python package repository. It is not required to clone the repo to your system or use it as a submodule.
• You no longer need to install protoc yourself. With version 4 a matching version of protoc is included.

Still to do:

• Distribute the embedded source and header files with the PyPi package. You still have to manually copy the src folder from the repo to your build system or include it in your own repo. Please do mind version mismatches.
• Documentation related to all these new capabilities.
• Other items related to the roadmap for v4.

How to
Installation
So the only requirement you need to have before installing Embedded Proto is Python 3. To then install Embedded Proto just type on the command line:
python3 -m pip install embeddedproto
This command may change slightly if you are on Windows. From the top of my head, it is:
python -m pip install embeddedproto
All matching requirements will be installed automatically.

My advice would be to either:

1. use a virtual environment (which you need to setup before installation see python venv docs), or
2. install for only the user (add the --user flag to the install command).

Usage
Next to generate your embedded proto message headers you no longer will call protoc directly. You have to call embeddedproto with similar parameters as protoc, here is an example:

embeddedproto -I. --eams_out=. simple_types.proto

• Note that you still have to provide one or multiple include folders with -I.
• You still have to provide the output folder with the plugin format of protoc --eams_out=XXX. This may become simpler before the actual release.
• Provide the proto file to process.
• Using the well-known types provided by protoc? To include those use the flag --IncludeWellKnownTypes.

As indicated before, the source files for the embedded proto library are not yet distributed with the package. For the time being please copy them from the develop-v4 branch on github.

I hope this is a solution to what you need. Looking forward to your feedback.