update to new mp spdz version #15
Conversation
grafjo
force-pushed
the
feature/upstream-new-mp-spdz
branch
from
1a8d0b8
to
dde0023
Compare
|
Needs to be rebased. |
Lila84
force-pushed
the
feature/upstream-new-mp-spdz
branch
3 times, most recently
from
84b5220
to
5a0e51b
Compare
kindlich
force-pushed
the
feature/upstream-new-mp-spdz
branch
3 times, most recently
from
2b7e095
to
7551467
Compare
|
@grafjo Thank you for submitting this pull request 👍 I have added a few comments and questions. Please have a look at the Codacy findings too. |
Codecov Report
@@ Coverage Diff @@
## master #15 +/- ##
==========================================
- Coverage 80.12% 79.75% -0.37%
==========================================
Files 29 29
Lines 1987 2010 +23
==========================================
+ Hits 1592 1603 +11
- Misses 335 342 +7
- Partials 60 65 +5
Continue to review full report at Codecov.
|
|
Before this PR can be merged, we would at least have to provide a carbynestack/MP-SPDZ base image with the desired version And if I remember correctly, reading inputs in MP-SPDZ has changed since version |
pkg/utils/os.go
Outdated
| err = command.Wait() | ||
| defer waitGroup.Done() | ||
| if err != nil { | ||
| println(fmt.Sprintf("Error occured!")) |
There was a problem hiding this comment.
We typically use a logger for emitting diagnosis information. What is the rationale of using println here instead? Is logging required here at all, as we anyway forward sysout/syserr to the caller?
There was a problem hiding this comment.
In this method we don't have a logger available.
The reason why we print here is that if we ran into a timeout for the request, sometimes the process output was not printed.
That is also the reason for using the wait-group here.
There was a problem hiding this comment.
You should be able to get a logger from the context using:
logger := log.FromContext(ctx)
There was a problem hiding this comment.
No longer relevant, removed the printlns.
|
Seems on commit is not signed-off properly. Needs to be fixed before we can merge. |
kindlich
force-pushed
the
feature/upstream-new-mp-spdz
branch
from
b623c6f
to
5c1f16e
Compare
@sbckr The compliant base image is made available via carbynestack/base-images and not via carbynestack/MP-SPDZ. The latter was used to provide an upstream patch for supporting PRNG seeds. Anyway, the respective Dockerfile has to be updated to use the new MP-SPDZ version. It can be set easily using a Docker build argument. Created cabynestack/base-images#2 to track this. In addition, the base image used by ko must be updated accordingly: Line 12 in 6e7113c |
|
Couple of open change requests (see conversation history) here:
|
|
Regarding the Task Info:
Differences from Stack-Deployment:
|
|
This PR needs another rebase to master. |
Co-authored-by: Petra Scherer <[email protected]> Co-authored-by: Timo Klenk <[email protected]> Signed-off-by: Johannes Graf <[email protected]> Signed-off-by: Petra Scherer <[email protected]> Signed-off-by: Timo Klenk <[email protected]>
Co-authored-by: Petra Scherer <[email protected]> Co-authored-by: Timo Klenk <[email protected]> Signed-off-by: Johannes Graf <[email protected]> Signed-off-by: Petra Scherer <[email protected]> Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Co-authored-by: Joo <[email protected]> Co-authored-by: Lila <[email protected]> Signed-off-by: Timo Klenk <[email protected]>
Co-authored-by: Joo <[email protected]> Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Co-authored-by: Lila <[email protected]> Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Careful: Requires an MP-SPDZ image that was built with `NO_CLIENT_TLS` Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
We don't need it if we properly handle the result on the caller-side Signed-off-by: Timo Klenk <[email protected]>
Signed-off-by: Timo Klenk <[email protected]>
kindlich
force-pushed
the
feature/upstream-new-mp-spdz
branch
from
8dbbf0d
to
f29277c
Compare
|
@grafjo @strieflin @kindlich Test was performed as follows:
git fetch origin pull/15/head:update-spdz
git checkout update-spdz
#
# Copyright (c) 2021 - for information on the respective copyright owner
# see the NOTICE file and/or the repository https://github.com/carbynestack/ephemeral.
#
# SPDX-License-Identifier: Apache-2.0
#
# defaultBaseImage: ghcr.io/carbynestack/ubuntu:20.04-20210827
# baseImageOverrides:
# github.com/carbynestack/ephemeral/cmd/ephemeral: ghcr.io/carbynestack/ephemeral-spdz-base-image:cleared-20210827
defaultBaseImage: ghcr.io/carbynestack/ubuntu:20.04-20210827-nonroot
baseImageOverrides:
github.com/carbynestack/ephemeral/cmd/ephemeral: ghcr.io/carbynestack/spdz:642d11f
ko publish -B --tags=spdz-update \
github.com/carbynestack/ephemeral/cmd/discovery \
github.com/carbynestack/ephemeral/cmd/ephemeral \
github.com/carbynestack/ephemeral/cmd/network-controller -L
kind load docker-image ko.local/network-controller:spdz-update --name apollo
kind load docker-image ko.local/network-controller:spdz-update --name starbuck
kind load docker-image ko.local/discovery:spdz-update --name apollo
kind load docker-image ko.local/discovery:spdz-update --name starbuck
kind load docker-image ko.local/ephemeral:spdz-update --name apollo
kind load docker-image ko.local/ephemeral:spdz-update --name starbuck
[...]
releases:
- name: {{ requiredEnv "RELEASE_NAME" }}-ephemeral
chart: carbynestack-oci/ephemeral
version: "0.1-SNAPSHOT-2275528139-15-7c3f0e0"
[...]
export APOLLO_FQDN="172.18.1.128.sslip.io"
export STARBUCK_FQDN="172.18.2.128.sslip.io"
export RELEASE_NAME=cs
export DISCOVERY_MASTER_HOST=$APOLLO_FQDN
export NO_SSL_VALIDATION=true
export DISCOVERY_IMAGE_REGISTRY=ko.local
export DISCOVERY_IMAGE_REPOSITORY=discovery
export DISCOVERY_IMAGE_TAG=spdz-update
export EPHEMERAL_IMAGE_REGISTRY=ko.local
export EPHEMERAL_IMAGE_REPOSITORY=ephemeral
export EPHEMERAL_IMAGE_TAG=spdz-update
export NETWORK_CONTROLLER_IMAGE_REGISTRY=ko.local
export NETWORK_CONTROLLER_IMAGE_REPOSITORY=network-controller
export NETWORK_CONTROLLER_IMAGE_TAG=spdz-update
export FRONTEND_URL=$STARBUCK_FQDN
export IS_MASTER=false
export AMPHORA_VC_PARTNER_URI=http://$APOLLO_FQDN/amphora
kubectl config use-context kind-starbuck
helmfile apply
export FRONTEND_URL=$APOLLO_FQDN
export IS_MASTER=true
export AMPHORA_VC_PARTNER_URI=http://$STARBUCK_FQDN/amphora
export CASTOR_SLAVE_URI=http://$STARBUCK_FQDN/castor
kubectl config use-context kind-apollo
helmfile apply
export CLI_VERSION=0.1-SNAPSHOT-1576571202-7-cf3db5b
curl -o cs.jar -L https://github.com/carbynestack/cli/releases/download/$CLI_VERSION/cli-$CLI_VERSION-jar-with-dependencies.jar
mkdir -p ~/.cs
cat <<EOF | envsubst > ~/.cs/config
{
"prime" : 198766463529478683931867765928436695041,
"r" : 141515903391459779531506841503331516415,
"noSslValidation" : true,
"trustedCertificates" : [ ],
"providers" : [ {
"amphoraServiceUrl" : "http://$apollo_fqdn/amphora",
"castorServiceUrl" : "http://$apollo_fqdn/castor",
"ephemeralServiceUrl" : "http://$apollo_fqdn/",
"id" : 1,
"baseUrl" : "http://$apollo_fqdn/"
}, {
"amphoraServiceUrl" : "http://$starbuck_fqdn/amphora",
"castorServiceUrl" : "http://$starbuck_fqdn/castor",
"ephemeralServiceUrl" : "http://$starbuck_fqdn/",
"id" : 2,
"baseUrl" : "http://$starbuck_fqdn/"
} ],
"rinv" : 133854242216446749056083838363708373830
}
EOF
curl -O -L https://github.com/carbynestack/base-images/raw/3595c5427915b2f9e1f22804e3f742cda9e72312/fake-crypto-material.zip
unzip -d crypto-material fake-crypto-material.zip
rm fake-crypto-material.zip
cat << 'EOF' > upload-tuples.sh
#!/bin/bash
SCRIPT_PATH="$( cd "$(dirname "$0")" ; pwd -P )"
TUPLE_FOLDER=${SCRIPT_PATH}/crypto-material/2-128-40
CLI_PATH=${SCRIPT_PATH}
NUMBER_OF_CHUNKS=1
function uploadTuples {
echo ${NUMBER_OF_CHUNKS}
for type in INPUT_MASK_GFP MULTIPLICATION_TRIPLE_GFP; do
for (( i=0; i<${NUMBER_OF_CHUNKS}; i++ )); do
local chunkId=$(uuidgen)
echo "Uploading ${type} to [http://${APOLLO_FQDN}/castor](http://$%7bAPOLLO_FQDN%7d/castor) (Apollo)"
java -jar ${CLI_PATH}/cs.jar castor upload-tuple -f ${TUPLE_FOLDER}/Triples-p-P0 -t ${type} -i ${chunkId} 1
local statusMaster=$?
echo "Uploading ${type} to [http://${STARBUCK_FQDN}/castor](http://$%7bSTARBUCK_FQDN%7d/castor) (Starbuck)"
java -jar ${CLI_PATH}/cs.jar castor upload-tuple -f ${TUPLE_FOLDER}/Triples-p-P1 -t ${type} -i ${chunkId} 2
local statusSlave=$?
if [[ "${statusMaster}" -eq 0 && "${statusSlave}" -eq 0 ]]; then
java -jar ${CLI_PATH}/cs.jar castor activate-chunk -i ${chunkId} 1
java -jar ${CLI_PATH}/cs.jar castor activate-chunk -i ${chunkId} 2
else
echo "ERROR: Failed to upload one tuple chunk - not activated"
fi
done
done
}
uploadTuples
EOF
chmod 755 upload-tuples.sh
./upload-tuples.sh
export JEFFS_NET_WORTH_ID=$(java -jar cs.jar amphora create-secret 177 -t billionaire=Jeff)
export ELONS_NET_WORTH_ID=$(java -jar cs.jar amphora create-secret 151 -t billionaire=Elon)
cat << 'EOF' > billionaires.mpc
# Prologue to read in the inputs
port=regint(10000)
listen(port)
socket_id = regint()
acceptclientconnection(socket_id, port)
v = sint.read_from_socket(socket_id, 2)
# The logic
first_billionaires_net_worth = v[0]
second_billionaires_net_worth= v[1]
result = first_billionaires_net_worth < second_billionaires_net_worth
# Epilogue to return the outputs
resp = Array(1, sint)
resp[0] = result
sint.write_to_socket(socket_id, resp)
EOF
export RESULT_ID=$(cat billionaires.mpc | java -jar cs.jar ephemeral execute \
-i $JEFFS_NET_WORTH_ID \
-i $ELONS_NET_WORTH_ID \
ephemeral-generic.default \
| tail -n +2 \
| sed 's/[][]//g')
java -jar cs.jar amphora get-secret $RESULT_IDI will provide pull requests to cover the required changes mentioned. However, this requires #15 to be merged |
pkg/utils/os.go
Outdated
| err = command.Wait() | ||
| defer waitGroup.Done() | ||
| if err != nil { | ||
| println(fmt.Sprintf("Error occured!")) |
There was a problem hiding this comment.
You should be able to get a logger from the context using:
logger := log.FromContext(ctx)
|
Thanks a lot @sbckr for testing this. That has been the last open point on our to-do list (see #15 (comment)). Ready to merge. |
requires #14
Updates to MP-SPDZ v0.2.8
corresponding cli part carbynestack/cli#19