Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add guide docs for rookify #688

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions docs/guides/configuration-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
---
sidebar_label: Rookify (technical preview)
sidebar_position: 31
---

# Configure Rookify: Migrate from Ceph Ansible to Rook (Technical Preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Config.yaml

The primary configuration file for Rookify is `config.yaml`. The repository contains an example file for general use, as well as one specifically tailored to the OSISM testbed setup:

- [config.example.yaml](https://github.com/SovereignCloudStack/rookify/blob/main/config.example.yaml)
- [config.example.osism.yaml](https://github.com/SovereignCloudStack/rookify/blob/main/config.example.osism.yaml)

### Parameters

#### General

```yaml title="config.example.yaml"
general:
machine_pickle_file: data.pickle
```

The general section allows for optional definition of a pickle file, which allows for saving the state of the migration as serialized objects on disk. The pickle filed can be named as pleased.

#### Logging

```yaml title="config.example.yaml"
logging:
level: INFO # level at which logging should start
format:
time: "%Y-%m-%d %H:%M.%S" # other example: "iso"
renderer: console # or: json
```

The `logging` section allows for specification of `structlog`. The `level` parameter can be set to all python [log-levels](https://docs.python.org/3/library/logging.html#logging-levels), i.e. `NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAl`, but it is recommended to use `INFO`.

#### Ceph

```yaml title="config.example.yaml"

ceph:
config: ./.ceph/ceph.conf
keyring: ./.ceph/ceph.client.admin.keyring
```

The `ceph` section specifies the `ceph.conf` and `ceph.client.admin.keyring` of the target systems (i.e. the system where Ceph-Ansible needs to be migrated to Rook).

#### SSH

```yaml title="config.example.yaml"
# fill in correct path to private key
ssh:
private_key: /home/USER/.ssh/cloud.private
hosts:
testbed-node-0:
address: 192.168.16.10
user: dragon
testbed-node-1:
address: 192.168.16.11
user: dragon
testbed-node-2:
address: 192.168.16.12
user: dragon
```

The `ssh` section requires specification of the `private_key` and `hosts`. The `hosts` section specifies the hostnames or aliases (e.g. keys like `testbed-node-0`), their ip-addresses (e.g. if rookify connects to the target systems per VPN add ips starting with `192.186...`) and user for login. If you are using the OSISM testbed, make sure that the private key does not contain any `EOF` or other strings, i.e. make sure the keys are 'clean' to avoid connection errors.

#### Kubernetes

```yaml title="config.example.yaml"

kubernetes:
config: ./k8s/config
```

The `kubernetes` section specifies the kubernetes configuration (e.g. if you use kubectl it is located in `~/.kube/config`) for rookifies kubernetes library. Rookify needs to connect to the kubernetes cluster on the target systems in order to use Rook.

```yaml title="config.example.yaml"
rook:
cluster:
name: osism-ceph
namespace: rook-ceph
ceph:
image: quay.io/ceph/ceph:v18.2.1
```

The `rook` sections requires some information about the Rook installation on the target system. Concerning the `cluster` Rookify needs to know the cluster-name and cluster-namespace. Rookify also needs to know the ceph version used by Rookify, i.e. the image version of the ceph container.

_Note_: Rookify does not install Rook for you. You need to provide a running Rook, i.e. a Rook Operator, on you target system.

_Note_: For OSISM specific migrations, Rookify needs to have some additional information, i.e. the respective labels for the rook resources:

```yaml title="config.example.osism.yaml"
rook:
cluster:
name: osism-ceph
namespace: rook-ceph
mds_placement_label: node-role.osism.tech/rook-mds
mgr_placement_label: node-role.osism.tech/rook-mgr
mon_placement_label: node-role.osism.tech/rook-mon
osd_placement_label: node-role.osism.tech/rook-osd
rgw_placement_label: node-role.osism.tech/rook-rgw
ceph:
image: quay.io/ceph/ceph:v18.2.1
```


#### Migration_modules

```yaml title="config.example.yaml"
migration_modules:
- analyze_ceph
- create_rook_cluster
- migrate_mons
- migrate_osds
- migrate_osd_pools
- migrate_mds
- migrate_mds_pools
- migrate_mgrs
- migrate_rgws
- migrate_rgw_pools
```

Rookify is designed to use a modular structure. It has contains various modules to migrate parts of Ceph-Ansible to Rook. The `migration_modules` section specifies which modules need to be run for the migration. Rookify contains more modules, take a look at the [`src/rookify/module`](https://github.com/SovereignCloudStack/rookify/tree/main/src/rookify/modules) directory to see the ones that are currently implemented.

_NOTE_: Many modules are dependent of each other. This means that some models will implicitly run other modules. For example: the `analyze_ceph` module specified above, will be run by all the modules. This means, that one does not need to specify it. It was added here only for reasons of clearity.
97 changes: 97 additions & 0 deletions docs/guides/deploy-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
sidebar_label: Rookify (technical preview)
sidebar_position: 51
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Deploy Rookify: Migrate to Rook from Ceph Ansible (technical preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

Currenlty Rookify is best install it on your own machine and then connected through vpn to you target system (the one where Ceph-Ansible needs to be rookified ;) ).

Rookify runs `in place`, which means that there are no parallel nodes necesssary. As mentioned above, Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_ but infrastructures are complicated, so precautionary backups and safety measure are very much advised.

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Prerequisites && Requirements

- A functioning Ceph cluster deployed via traditional methods.
- Access to a Kubernetes cluster with sufficient resources to host the
migrated Ceph cluster.
- Kubernetes nodes should be rolled out at least on the OSD nodes
- Rook operator version 1.13 or higher installed in the Kubernetes clu
ster.
- _local development enivornment_ requires radoslib version 2.0.0 inst
alled

## Manual Installation

### Download or Clone the Repository

Clone or download Rookify from the [repository](https://github.com/SovereignCloudStack/rookify).

## Install and Run Locally

1. Navigate to the tool directory:

```bash
cd rookify
```

2. To install Rookify locally, pythons virtualenv will be used (Note: This will install pre-commit in your local user context):

:::tip

Checkout the included options of the added `Makefile` by simply typing `make`.

:::

```
make setup
```

This command also checks if you have the required python library for `radoslib` installed. Make sure to have it installed on your linux distribution.

:::tip

Before running rookify, first check all options by using `rookify --help`

:::

To run rookify you can either run it directly from within pythons virtualenv or with help of the make file:

```bash
# directly
./.venv/bin/rookify --help
# using make
make run-local-rookify
```

## Install and Run from within a Container

1. Navigate to the tool directory:

2. To install Rookify into a container, podman or docker can be used (Note: in both cases pythons library for `radoslib` needs to be installed locally):

```
make check-radoslib
make up
```

This command uses `docker compose`, so make sure you have that installed as well.

To run rookify you can either enter the container and run rookify from there or use `make run-rookify`.

:::note

Before running rookify, it can be usefull to check all options by using `rookify --help`

:::
17 changes: 17 additions & 0 deletions docs/guides/deploy-guide/services/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
sidebar_label: Migrate Ceph-Ansible via Rookify (technical preview)
sidebar_position: 51
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Migrate Ceph-Ansible via Rookify (technical preview)

:::warning

This is a technical preview and not recommended for production use yet.

:::

Rookify is now also available in OSISM.
83 changes: 83 additions & 0 deletions docs/guides/operations-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
sidebar_label: Rookify (technical preview)
---

# Use Rookify: Migrate to Rook from Ceph Ansible (technical preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## Consider using a pickle file

In order to have a trackable state of process you can add a pickle file. Specify this option in the `config.yaml`:

```yaml title="config.example.yaml"
general:
machine_pickle_file: data.pickle
```

Now you will be able to to view the state of progress by running `rookify --show-state`.

:::warning
Rookify will take data.pickle as a source of truth for its operations. If you want to start a clean migration, be sure to delete the file.
:::

## Rookify CLI

### Run

:::warning
Currenlty rookify executes per default
:::

Currenlty rookify executes per default. This means: if you run rookify like so `.venv/bin/rookify`, it will start the migration as configured in `config.yaml`.

### --dry-run

:::tip
Run preflight-mode to check if Rookify can connect to your target systems
:::

Rookify has a `preflight-mode` to check if basic commands and the connection to the target systems are working.
You can always run `--dry-run` mode, without any danger that migration processes will be executed.

### --help

Run `--help` to see the various options the CLI provides.

### --show

Run `--show` if you used a pickel-file (see the configuration-guide concerning the general section). This will show the status of your migration process.

## Debugging and Testing

If you suspect that something is not working in Rookify itself, you can start setting logging to `DEBUG`.

If you suspect that something is not working in Rookify itself, you could start by running the tests. For this you need to have access to rookifies code.

### Set logging to debug level

Edit the `config.yaml` and set level to "DEBUG":

```yaml title="config.example.yaml"
logging:
level: DEBUG
format:
time: "%Y-%m-%d %H:%M.%S" # other example: "iso"
renderer: console # or: json
```

You can also set the other formatting options, as indicated by the comments, for further options take a look a the documentation of [structlog](https://www.structlog.org/en/stable/standard-library.html).

### Run tests

Make sure you can reach Rookifies code on you system. Then you can either:

1. run `make run-tests-locally` from the working directory of the rookify repository. If you prefer to use a containerized setup, use `make run-tests`.
2. run `.venv/bin/python -m pytest` from the virtual environment of you installation
32 changes: 32 additions & 0 deletions docs/guides/troubleshooting-guide/rookify.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
sidebar_label: Rookify (technical preview)
sidebar_position: 40
---

# Rookify (technical preview)

:::warning

Rookify is developed to migrate from Ceph-Ansible to Rook _in place_ and _without downtime_.
Nevertheless, it is **strongly advised** to test Rookify in a controlled environment beforehand, such as the [OSISM testbed](https://github.com/osism/testbed). Additionally, ensure that all precautionary backups are taken, and any other necessary safety measures are in place.

:::

The [Rookify GitHub repository](https://github.com/SovereignCloudStack/rookify) includes a README.md that provides a condensed summary of the information covered here.

## ssh-issues


**"Failed to load private key" **


- make sure the id-rsa keys are "clean" and do not contain unexpected strings like "\<\<EOF". Possible solution: Clean the keys manually or call `ssh-keygen -p -N "" -f ssh.key` to convert and reformat the keyfile to the expected format.


**"too many authentications error"**

- Sometimes too many keys loaded are loaded by the ssh-agent. Possible solution: Switch off the ssh-agent on you machine. You can do this yourself or by allowing direnv ( `direnv allow` ) to use `.envrc`.

## frozen state

- if the rookify process freezes, check your connections. In the OSISM testbed especially check the vpn-connection (in testbed epository try `make vpn-*`)
Loading