forked from hyperledger/cacti
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(persistence-fabric): add sample setup scripts, improve documenta…
…tion
- Fix error when empty `transaction.actions` was received from a block.
- Copy schema SQL during build.
- Remove unique constraint on fabric transactions hash column (there
are transactions with empty hash that would break it).
- Improve parsing of certificate subject/issuer attributes to handle more
delimiters.
- Use new `install-fabric.sh` script (that is recommended for Fabric 2.5)
instead of old bootstrap to fix some runtime issues that I've encountered.
- Add sample setup scripts. Simple can be used to run persistence against
already running fabric ledger, complete will setup entire environment
and run some basic operations to generate sample data.
- Improve documentation to include these new scripts and how to use them, fix
smaller issues.
Signed-off-by: Michal Bajer <[email protected]>- Loading branch information
Showing
12 changed files
with
552 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,53 +25,94 @@ Clone the git repository on your local machine. Follow these instructions that w | |
|
|
||
| ### Prerequisites | ||
|
|
||
| #### Build | ||
|
|
||
| In the root of the project, execute the command to install and build the dependencies. It will also build this persistence plugin: | ||
|
|
||
| ```sh | ||
| yarn run configure | ||
| ``` | ||
|
|
||
| ### Usage | ||
| #### Hyperledger Fabric Ledger and Connector | ||
|
|
||
| Instantiate a new `PluginPersistenceFabric` instance: | ||
| This plugin requires a running Hyperledger Fabric ledger that you want to persist to a database. For testing purposes, you can use our [test fabric-all-in-one Docker image](../../tools/docker/fabric-all-in-one/README.md). To access the ledger you'll need your organization connection profile JSON and a wallet containing registered identity. If you are using our `fabric-all-in-one` image, you can run our [asset-transfer-basic-utils scripts](../../tools/docker/fabric-all-in-one/asset-transfer-basic-utils/README.md) to fetch Org1 connection profile from a docker and register new user to a localhost wallet. | ||
|
|
||
| ```typescript | ||
| import { PluginPersistenceFabric } from "@hyperledger/cactus-plugin-persistence-fabric"; | ||
| import { v4 as uuidv4 } from "uuid"; | ||
| ```shell | ||
| # Start the test ledger | ||
| docker compose -f tools/docker/fabric-all-in-one/docker-compose-v2.x.yml up | ||
| # Wait for it to start (status should become `healthy`) | ||
|
|
||
| const persistencePlugin = new PluginPersistenceFabric({ | ||
| apiClient, | ||
| logLevel: "info", | ||
| instanceId, | ||
| connectionString: "postgresql://postgres:your-super-secret-and-long-postgres-password@localhost:5432/postgres",, | ||
| channelName: "mychannel", | ||
| gatewayOptions: { | ||
| identity: signingCredential.keychainRef, | ||
| wallet: { | ||
| keychain: signingCredential, | ||
| }, | ||
| }, | ||
| }); | ||
| # Run asset-transfer-basic-utils scripts | ||
| cd tools/docker/fabric-all-in-one/asset-transfer-basic-utils | ||
| # Cleanup artifacts from previous runs | ||
| rm -fr wallet/ connection.json | ||
| # Fetch connection profile to `tools/docker/fabric-all-in-one/asset-transfer-basic-utils/connection.json` | ||
| # Enroll user using wallet under `tools/docker/fabric-all-in-one/asset-transfer-basic-utils/wallet` | ||
| npm install | ||
| CACTUS_FABRIC_ALL_IN_ONE_CONTAINER_NAME=fabric_all_in_one_testnet_2x ./setup.sh | ||
| ``` | ||
|
|
||
| // Initialize the connection to the DB | ||
| await persistencePlugin.onPluginInit(); | ||
| Once you have an Fabric ledger ready, you need to start the [Ethereum Cacti Connector](../cactus-plugin-ledger-connector-fabric/README.md). We recommend running the connector on the same ApiServer instance as the persistence plugin for better performance and reduced network overhead. See the connector package README for more instructions, or check out the [setup sample scripts](./src/test/typescript/manual). | ||
|
|
||
| #### Supabase Instance | ||
|
|
||
| You need a running Supabase instance to serve as a database backend for this plugin. | ||
|
|
||
| ### Setup Tutorials | ||
|
|
||
| We've created some sample scripts to help you get started quickly. All the steps have detailed comments on it so you can quickly understand the code. | ||
|
|
||
| #### Sample Setup | ||
|
|
||
| Location: [./src/test/typescript/manual/sample-setup.ts](./src/test/typescript/manual/sample-setup.ts) | ||
|
|
||
| This sample script can be used to set up `ApiServer` with the Fabric connector and persistence plugins to monitor and store ledger data in a database. You need to have a ledger running before executing this script. | ||
|
|
||
| To run the script you need to set the following environment variables: | ||
|
|
||
| - `FABRIC_CONNECTION_PROFILE_PATH`: Full path to fabric ledger connection profile JSON file. | ||
| - `FABRIC_CHANNEL_NAME`: Name of the channel we want to connect to (to store it's data). | ||
| - `FABRIC_WALLET_PATH` : Full path to wallet containing our identity (that can connect and observe specified channel). | ||
| - `FABRIC_WALLET_LABEL`: Name (label) of our identity in a wallet provided in FABRIC_WALLET_PATH | ||
|
|
||
| By default, the script will try to use our `supabase-all-in-one` instance running on localhost. This can be adjusted by setting PostgreSQL connection string in `SUPABASE_CONNECTION_STRING` environment variable (optional). | ||
|
|
||
| ```shell | ||
| # Example assumes fabric-all-in-one was used. Adjust the variables accordingly. | ||
| FABRIC_CONNECTION_PROFILE_PATH=/home/cactus/tools/docker/fabric-all-in-one/asset-transfer-basic-utils/connection.json FABRIC_CHANNEL_NAME=mychannel FABRIC_WALLET_PATH=/home/cactus/tools/docker/fabric-all-in-one/asset-transfer-basic-utils/wallet FABRIC_WALLET_LABEL=appUser | ||
| node ./dist/lib/test/typescript/manual/sample-setup.js | ||
| ``` | ||
|
|
||
| Alternatively, import `PluginFactoryLedgerPersistence` from the plugin package and use it to create a plugin. | ||
| #### Complete Sample Scenario | ||
|
|
||
| Location: [./src/test/typescript/manual/common-setup-methods](./src/test/typescript/manual/common-setup-methods) | ||
|
|
||
| This script starts the test Hyperledger Fabric ledger for you and executes few transactions on a `basic` chaincode. Then, it synchronizes everything to a database and monitors for all new blocks. This script can also be used for manual, end-to-end tests of a plugin. | ||
|
|
||
| By default, the script will try to use our `supabase-all-in-one` instance running on localhost. | ||
|
|
||
| ```shell | ||
| npm run complete-sample-scenario | ||
| ``` | ||
|
|
||
| Custom supabase can be set with environment variable `SUPABASE_CONNECTION_STRING`: | ||
|
|
||
| ```shell | ||
| SUPABASE_CONNECTION_STRING=postgresql://postgres:[email protected]:5432/postgres npm run complete-sample-scenario | ||
| ``` | ||
|
|
||
| ### Usage | ||
|
|
||
| Instantiate a new `PluginPersistenceFabric` instance: | ||
|
|
||
| ```typescript | ||
| import { PluginFactoryLedgerPersistence } from "@hyperledger/cactus-plugin-persistence-fabric"; | ||
| import { PluginImportType } from "@hyperledger/cactus-core-api"; | ||
| import { PluginPersistenceFabric } from "@hyperledger/cactus-plugin-persistence-fabric"; | ||
| import { v4 as uuidv4 } from "uuid"; | ||
|
|
||
| const factory = new PluginFactoryLedgerPersistence({ | ||
| pluginImportType: PluginImportType.Local, | ||
| }); | ||
|
|
||
| const persistencePlugin = await factory.create({ | ||
| apiClient, | ||
| const persistencePlugin = new PluginPersistenceFabric({ | ||
| apiClient: new FabricApiClient(apiConfigOptions), | ||
| logLevel: "info", | ||
| instanceId, | ||
| instanceId: "my-instance", | ||
| connectionString: "postgresql://postgres:your-super-secret-and-long-postgres-password@localhost:5432/postgres",, | ||
| channelName: "mychannel", | ||
| gatewayOptions: { | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
171 changes: 171 additions & 0 deletions
171
packages/cactus-plugin-persistence-fabric/src/test/typescript/manual/common-setup-methods.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| /** | ||
| * Common setup code for the persistence plugin with detailed comments on each step. | ||
| * Requires environment variable `SUPABASE_CONNECTION_STRING` to be set before running the script that includes this! | ||
| * If not provided, a localhost instance of supabase will be assumed. | ||
| */ | ||
|
|
||
| import process from "process"; | ||
| import { v4 as uuidV4 } from "uuid"; | ||
| import { PluginRegistry } from "@hyperledger/cactus-core"; | ||
| import { PluginKeychainMemory } from "@hyperledger/cactus-plugin-keychain-memory"; | ||
| import { | ||
| LogLevelDesc, | ||
| LoggerProvider, | ||
| Logger, | ||
| } from "@hyperledger/cactus-common"; | ||
| import { Configuration } from "@hyperledger/cactus-core-api"; | ||
| import { | ||
| ApiServer, | ||
| AuthorizationProtocol, | ||
| ConfigService, | ||
| } from "@hyperledger/cactus-cmd-api-server"; | ||
| import { DiscoveryOptions, X509Identity } from "fabric-network"; | ||
| import { | ||
| DefaultEventHandlerStrategy, | ||
| FabricApiClient, | ||
| PluginLedgerConnectorFabric, | ||
| } from "@hyperledger/cactus-plugin-ledger-connector-fabric"; | ||
|
|
||
| import { PluginPersistenceFabric } from "../../../main/typescript"; | ||
|
|
||
| ////////////////////////////////// | ||
| // Constants | ||
| ////////////////////////////////// | ||
|
|
||
| const SUPABASE_CONNECTION_STRING = | ||
| process.env.SUPABASE_CONNECTION_STRING ?? | ||
| "postgresql://postgres:[email protected]:5432/postgres"; | ||
|
|
||
| const testLogLevel: LogLevelDesc = "info"; | ||
| const sutLogLevel: LogLevelDesc = "info"; | ||
|
|
||
| // Logger setup | ||
| const log: Logger = LoggerProvider.getOrCreate({ | ||
| label: "common-setup-methods", | ||
| level: testLogLevel, | ||
| }); | ||
|
|
||
| /** | ||
| * Common ApiServer instance, can be empty if setup was not called yet! | ||
| */ | ||
| let apiServer: ApiServer; | ||
|
|
||
| ////////////////////////////////// | ||
| // Methods | ||
| ////////////////////////////////// | ||
|
|
||
| /** | ||
| * Setup Cacti ApiServer instance containing Fabric Connector plugin (for accessing the fabric ledger) | ||
| * and Fabric Persistence plugin (for storing data read from ledger to the database). | ||
| * | ||
| * @param port Port under which an ApiServer will be started. Can't be 0. | ||
| * @param channelName Channel that we want to connect to. | ||
| * @param connectionProfile Fabric connection profile (JSON object, not a string!) | ||
| * @param userIdentity Signing identity to use to connect to the channel (object, not a string!) | ||
| * | ||
| * @returns `{ persistence, apiClient, signingCredential }` | ||
| */ | ||
| export async function setupApiServer( | ||
| port: number, | ||
| channelName: string, | ||
| connectionProfile: any, | ||
| userIdentity: X509Identity, | ||
| ) { | ||
| // PluginLedgerConnectorFabric requires a keychain plugin to operate correctly, ensuring secure data storage. | ||
| // We will store our userIdentity in it. | ||
| // For testing and debugging purposes, we use PluginKeychainMemory, which stores all secrets in memory (remember: this is not secure!). | ||
| const keychainId = uuidV4(); | ||
| const keychainEntryKey = "monitorUser"; | ||
| const keychainPlugin = new PluginKeychainMemory({ | ||
| instanceId: uuidV4(), | ||
| keychainId, | ||
| backend: new Map([[keychainEntryKey, JSON.stringify(userIdentity)]]), | ||
| logLevel: testLogLevel, | ||
| }); | ||
| const signingCredential = { | ||
| keychainId, | ||
| keychainRef: keychainEntryKey, | ||
| }; | ||
|
|
||
| // We create fabric connector instance with some default settings assumed. | ||
| const discoveryOptions: DiscoveryOptions = { | ||
| enabled: true, | ||
| asLocalhost: true, | ||
| }; | ||
| const connector = new PluginLedgerConnectorFabric({ | ||
| instanceId: uuidV4(), | ||
| pluginRegistry: new PluginRegistry({ plugins: [keychainPlugin] }), | ||
| sshConfig: {}, | ||
| cliContainerEnv: {}, | ||
| peerBinary: "/fabric-samples/bin/peer", | ||
| logLevel: sutLogLevel, | ||
| connectionProfile, | ||
| discoveryOptions, | ||
| eventHandlerOptions: { | ||
| strategy: DefaultEventHandlerStrategy.NetworkScopeAnyfortx, | ||
| commitTimeout: 300, | ||
| }, | ||
| }); | ||
|
|
||
| // Remember to initialize a plugin | ||
| await connector.onPluginInit(); | ||
|
|
||
| // We need an `FabricApiClient` to access `PluginLedgerConnectorFabric` methods from our `PluginPersistenceFabric`. | ||
| const apiConfig = new Configuration({ basePath: `http://127.0.0.1:${port}` }); | ||
| const apiClient = new FabricApiClient(apiConfig); | ||
|
|
||
| // We create persistence plugin, it will read data from fabric ledger through `apiClient` we've just created, | ||
| // and push it to PostgreSQL database accessed by it's SUPABASE_CONNECTION_STRING (read from the environment variable). | ||
| const persistence = new PluginPersistenceFabric({ | ||
| channelName, | ||
| gatewayOptions: { | ||
| identity: signingCredential.keychainRef, | ||
| wallet: { | ||
| keychain: signingCredential, | ||
| }, | ||
| }, | ||
| apiClient, | ||
| logLevel: sutLogLevel, | ||
| instanceId: uuidV4(), | ||
| connectionString: SUPABASE_CONNECTION_STRING, | ||
| }); | ||
| // Plugin initialization will check connection to the database and setup schema if needed. | ||
| await persistence.onPluginInit(); | ||
|
|
||
| // The API Server is a common "container" service that manages our plugins (connector and persistence). | ||
| // We use a sample configuration with most security measures disabled for simplicity. | ||
| log.info("Create ApiServer..."); | ||
| const configService = new ConfigService(); | ||
| const cactusApiServerOptions = await configService.newExampleConfig(); | ||
| cactusApiServerOptions.authorizationProtocol = AuthorizationProtocol.NONE; | ||
| cactusApiServerOptions.configFile = ""; | ||
| cactusApiServerOptions.apiCorsDomainCsv = "*"; | ||
| cactusApiServerOptions.apiTlsEnabled = false; | ||
| cactusApiServerOptions.apiPort = port; | ||
| const config = await configService.newExampleConfigConvict( | ||
| cactusApiServerOptions, | ||
| ); | ||
|
|
||
| apiServer = new ApiServer({ | ||
| config: config.getProperties(), | ||
| pluginRegistry: new PluginRegistry({ plugins: [connector, persistence] }), | ||
| }); | ||
|
|
||
| const apiServerStartOut = await apiServer.start(); | ||
| log.debug(`apiServerStartOut:`, apiServerStartOut); | ||
| // Our setup is operational now! | ||
|
|
||
| return { persistence, apiClient, signingCredential }; | ||
| } | ||
|
|
||
| /** | ||
| * Cleanup all the resources allocated by our Api Server. | ||
| * Remember to call it before exiting! | ||
| */ | ||
| export async function cleanupApiServer() { | ||
| log.info("cleanupApiServer called."); | ||
|
|
||
| if (apiServer) { | ||
| await apiServer.shutdown(); | ||
| } | ||
| } |
Oops, something went wrong.