This project is a trial to train a neural network to play Facebook's Jumpy Jumpy Game
The full source code is available under GPL-3.0 License in my Github repository ghaiszaher/JumpyNN
Make sure that the following are installed:
- Python3
- Keras
- Tensorflow
- Opencv2 & Numpy
- Pillow
- pynput
- mss
- Jupyter Notebook
For this version, auto-calibration was not made. You need to open Facebook's Jumpy Jumpy Game and go to the file "jumpy_settings" to adjust the following:
- monitor.left & monitor.top: the location of the game's div relative to the screen
- monitor.width & monitor.height: the size of the game's div
Template files are responsible for determining wether the current screen is Main Menu, Scoreboard or Continue Playing. They are located in templates folder:
You need to replace these 3 images with images taken from your screen inside the gameplay.
From inside the game, make sure to adjust the ball color to the Pink one, otherwise you will need to change the colors settings.
Run the file "record.py" and play the game to collect images & outputs for your gameplay. A new folder will be created "gameplay_logs" that contains images + mouse dragging movement info during the gameplay.
- Extract features from images and generate Training Data: Run the jupyter notebook 's file generate_X_y. This will generate several files called "features_X_y.npz", the final one will be in "gameplay_logs/features_X_y.npz", and will contain the training data from all the images combined.
- Run the jupyter notebook's file "train.ipynb" to create and run the neural network. Final Neural Network files (config & weights) will be stored in "gameplay_logs"
Run the file "run.py" to test
Trained network's gameplay:
If you're trying to run this on Windows, you might have some issues with mouse drag using pyautogui library. I tested it on Ubuntu 18.04.2 LTS.