Support project:
Estuary docker deployer service which will run your containers in docker or K8s with kubectl.
Deployer service can run your commands using the estuary-agent service. Check-out the wiki !
- Debug accelerator. No more logs attached to Jira. QAs can push the related bug-ish image to registry. Then in a docker compose QA can deploy "the bug" on a developer's machine or on dedicated debug machine (security reason due to docker sock mounting). The dev will have all he needs
- Testing as a service. A complete enevironment SUT/BD/Automation Framework can be deployed and tested
- Bring up Docker containers remotely through HTTP
- Templating with Jinja2
docker-compose up
docker network create estuarydeployer_default
docker run -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock --net=estuarydeployer_default \
estuaryoss/deployer:<tag>
docker run -p 8080:8080 -d --privileged --name some-docker -d docker:dind
docker exec -i -t some-docker docker login
docker exec -i -t some-docker docker run -p 8080:8080 -ti --net=deployer estuaryoss/deployer:latest
docker exec -i -t some-docker docker run -p 8080:8080 -ti --net=deployer -v /var/run/docker.sock:/var/run/docker.sock \
estuaryoss/deployer:latest
To have all your deployer instances in a central location we use netflix eureka. This means your client will discover all services used for your test and then spread the tests across all.
Start Eureka server with docker:
docker run -p 8080:8080 estuaryoss/netflix-eureka:1.9.25Start your containers by specifying the full hostname or ip of the host machine on where your deployer service resides.
docker network create estuarydeployer_default
docker run params:
Optional:
-e MAX_DEPLOYMENTS=3 -> how many deployments to be done (docker only). Default is 10.
-e EUREKA_SERVER="http://10.13.14.28:8080/eureka/v2" -> eureka server
-e APP_IP_PORT="10.13.14.28:8081" -> the app hostname/ip:port. Mandatory if EUREKA_SERVER is used
-e APP_APPEND_LABEL="lab" -> id will be appended to the default app name on service registration. Useful for user mappings service-resources on a VM
-e FLUENTD_IP_PORT="10.13.14.28:24224" -> fluentd log collector agent target ip:port
-e ENV_EXPIRE_IN=1440 -> [minutes] How long it will take before the env will be deleted. Default is 1440 minutes.
-e SCHEDULER_POLL_INTERVAL=120 -> [seconds] The env expire scheduler polling interval. Default is 1200 seconds.
-e DEPLOY_WITH=kubectl -> [docker/kubectl] The deployment environment. Default is docker.
Mandatory:
-p 8081:8080 -> port fwd from docker 8080 to host 8081
-v /var/run/docker.sock:/var/run/docker.sock -> docker sock mount
--net=estuarydeployer_default -> bind to this net prior created
estuaryoss/deployer:latest
Please consult Fluentd for logging setup.
Estuary-deployer tags all logs in format estuary-deployer.**
Matcher example:
<match estuary*.**>
@type stdout
</match>Run example:
docker network create estuarydeployer_default
docker run \
-e FLUENTD_IP_PORT=10.10.10.1:24224 \
-p 8080:8080
-v /var/run/docker.sock:/var/run/docker.sock \
--net=estuarydeployer_default \
estuaryoss/deployer:<tag>
For auth set HTTP_AUTH_TOKEN env variable.
!!! Use this env variable, otherwise you will open a major security hole. The attacker will have access to your system. !!!
Run example:
docker run \
-e HTTP_AUTH_TOKEN=change_this_secret
-p 8080:8080
estuaryoss/deployer:<tag>
Then, access the Http Api. Call example:
curl -i -H 'Token:mysecret' http:localhost:8080/about
Set HTTPS_ENABLE env var option to true or false.
Set the certificate and the private key path with HTTPS_CERT and HTTPS_KEY env variables.
If you do not set cert and private key file env vars, it defaults to a folder in the same path called https, and the default files https/cert.pem and https/key.pem.
User defined environment variables will be stored in a 'virtual' environment. The extra env vars will be used by the process that executes system commands.
There are two ways to inject user defined environment variables.
- call POST on /env endpoint. The body will contain the env vars in JSON format. E.g. {"FOO1":"BAR1"}
- create an environment.properties file with the extra env vars needed and place it in the same path as the JAR. Example in this repo.
! All environment variables described above can also be set using environment.properties.
pyinstaller --onefile --clean --add-data="rest/api/views/swaggerui/:rest/api/views/swaggerui/" main.pyThe user can put any metadata under x-metadata section.
The Deployer service does not enforce any metadata standardization.
However, it's a good practice to have:
- the deployment name: name
- the labels for the deployment: labels
The two fields are used by seeder in order to organize and orchestrate the deployments.
Obs: The x-metadata section is supported in compose version >= 3.3.
version: "3.3"
x-metadata:
name: alpine-test
labels:
k1: v1
k2: v2
services:
alpine:
restart: always
image: alpine:3.9.4
hostname: alpine
entrypoint: tail -f /etc/hostnamecurl http://172.17.0.22:8083/docker/deployments
{
"code" : 1000,
"description" : [
{
"containers" : [
"346dd96ed55a estuaryoss/discovery:4.0.8 \"/scripts/main_flask…\" 16 seconds ago Up 15 seconds 8080/tcp 6961ca05296ce48d_container_1"
],
"id" : "6961ca05296ce48d"
}
],
"message" : "Success",
"name" : "estuary-deployer",
"timestamp" : "2020-08-15 20:32:25.971933",
"path" : "/deployments?",
"version" : "4.0.8"
}http://192.168.100.12:8083/kubectl/ping
http://192.168.100.12:8083/docker/ping
Estuary deployer
Estuary agent
Estuary discovery
Estuary viewer