Remember when machine learning was something only data scientists could do? Those days are long gone. Thanks to tools like Apple’s CreateML, anyone with a Mac can create their own machine learning models in minutes.

In this tutorial, we’ll build a hand gesture recognition model that can distinguish between thumbs up and peace signs — no coding required!

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2Fcreate-machine-learning-model-no-code%2F6fde2a63-ef35-4fe6-95ff-fb259a443ce4.png?alt=media&token=7461dd64-ee64-4df0-8eab-f588f31dff80","id":"6fde2a63-ef35-4fe6-95ff-fb259a443ce4","width":3840,"height":2400,"filename":"mediumMLpost Cover.png","type":"image/png","caption":"Let’s train a model in less than 10 min","border":false}]}
```

### What We’ll Build

A simple machine learning model that can recognize two hand gestures:

* 👍 Thumbs up

* ✌️ Peace sign

### What You’ll Need

* A Mac with Photo Booth (pre-installed | Free)

* [Xcode (includes CreateML | Free)](https://developer.apple.com/xcode/)

* About 10 minutes of your time

* 👋 Your hands!

## Step 1: Capturing Your Training Data

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2FwT9SvFAz3LFcSoxH5CKW%2F933507f2-01b6-4e72-a78c-ab8c0715d529.png?alt=media&token=0e177c23-9226-438d-ad7c-aa07d1d166d5","id":"933507f2-01b6-4e72-a78c-ab8c0715d529","width":3840,"height":2400,"filename":"mediumMLpost 01.png","type":"image/png","caption":"Take at least 10 pictures of the gesture you want to classify.","border":false}]}
```

1. Open Photo Booth on your Mac

2. Position yourself where there’s good lighting

3. Take around 10 photos of each gesture:

* 10 photos doing thumbs up

* 10 photos making peace signs

*TIP: Vary your hand position slightly in each photo for better results*

## Step 2: Setting Up Your Folders

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2FwT9SvFAz3LFcSoxH5CKW%2Fbea37a2b-10f4-4198-ae72-6a6c48bdda02.png?alt=media&token=adf18a5e-9c75-457c-a406-1fb97a1245ad","id":"bea37a2b-10f4-4198-ae72-6a6c48bdda02","width":3840,"height":2400,"filename":"mediumMLpost 2.png","type":"image/png","caption":"The folders are the classifications we want to train the model for.","border":false}]}
```

Create this folder structure on your Mac:

```
Hand Gestures
├── training
│   ├── thumbs up
│   └── peace
└── testing
    ├── thumbs up
    └── peace
```

Quick way to do this:

1. Create a main folder called “Hand Gestures”

2. Inside it, create two folders: “training” and “testing”

3. Inside both training and testing, create two folders: “thumbs up” and “peace”

## Step 3: Organizing Your Photos

1. From your Photo Booth library, select your photos

2. For each gesture, distribute your photos:

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2Fcreate-machine-learning-model-no-code%2F75e84e50-2b28-4877-92eb-c1eda4161dfa.png?alt=media&token=f37841b4-e26c-42d1-8520-0d3e679922b8","id":"75e84e50-2b28-4877-92eb-c1eda4161dfa","width":3840,"height":2400,"filename":"mediumMLpost 3.png","type":"image/png","caption":"For the tutorial we are giving 5 picture for training and testing.","border":false}]}
```

* Put 5 thumbs up photos in training/thumbs up

* Put 5 peace sign photos in training/peace

* Put the remaining photos in the corresponding testing folders

*Pro Tip: Choose your best photos for the training set!*

## Step 4: Train Your Model

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2FwT9SvFAz3LFcSoxH5CKW%2Fcf6da33a-3c1b-401a-aad0-52284d83b245.mp4?alt=media&token=b9b5f430-49f7-4f38-8833-f0e3df5b3d69","id":"cf6da33a-3c1b-401a-aad0-52284d83b245","width":1764,"height":1080,"filename":"Train.mp4","type":"video/mp4","caption":"Now add your trained and testing folders to CreateML and click on train.","border":false}]}
```

1. Open CreateML (Use Spotlight or you can find it in Xcode > Xcode menu > Open Developer Tool > Create ML)

2. Click “New Document”

3. Choose “Hand Pose Classification”

4. In the Training Data section:

* Drag your “training” folder onto the Training Data box

* Drag your “testing” folder onto the Testing Data box

5\. Click the “Train” button and watch the magic happen!

### **Testing Your Model**

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2FwT9SvFAz3LFcSoxH5CKW%2F5ca9339e-9bed-4a14-b728-6086a02d0e34.mp4?alt=media&token=2f39771a-2529-4c33-8c9c-3a8bc9aa0f98","id":"5ca9339e-9bed-4a14-b728-6086a02d0e34","width":1764,"height":1080,"filename":"Test.mp4","type":"video/mp4","caption":"Click on the preview tab to test your model in CreateML","border":false}]}
```

Click on the preview tab to test your model.

Once training is complete:

1. Look at the evaluation metrics on the right

2. Try the Preview tab to test your model with your webcam

3. Make some gestures and see how well it recognizes them!

### **Improving Your Results**

Not getting perfect results? \
Try these tips:

* Take more photos with different: Hand positions, Lighting conditions, Backgrounds and/or angles

* Try playing with the “Augmentations” in the “Train” view.

* Make sure your hands are clearly visible in all photos

* Remove any blurry or unclear photos

* Adding more images to the dataset

* Add an “Empty” state. This is a classification with out any gestures.

### **Exporting Your Model**

You can also export the model by clicking the “Get” icon.

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2Fcreate-machine-learning-model-no-code%2F4751d0b9-26fb-4c41-b2ce-754fb684a126.png?alt=media&token=3fb9afa4-16a5-4dea-bea3-736b8b15ff53","id":"4751d0b9-26fb-4c41-b2ce-754fb684a126","width":3840,"height":2400,"filename":"mediumMLpost 5.png","type":"image/png","caption":"You can also export the model by clicking the “Get” icon.","border":false}]}
```

Happy with the results?

1. Click “File > Export”

2. Save your model as “HandGestureClassifier.mlmodel”

3. You can now use this model in your iOS or macOS apps!

## **What’s Next?**

Now that you’ve created your first model, here are some exciting ways to build upon this knowledge:

### **Try ML Shortcuts**

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FACjrUf7Iysghkn0bdJ5MEZs1Yss1%2FwT9SvFAz3LFcSoxH5CKW%2F40cd1d56-a93a-4d4f-b3f8-751105628d1d.mp4?alt=media&token=7b0057b8-40b1-45a9-8f51-ac310925faca","id":"40cd1d56-a93a-4d4f-b3f8-751105628d1d","width":1085,"height":1080,"filename":"copy_41783297-F026-448D-ACD8-728E49647573 copy.mp4","type":"video/mp4","caption":"Early demo of using the app to control a Figma prototype.","border":false}]}
```

Want to see your hand gesture model in action right away? [Check out ML Shortcuts ](https://www.annotate.supply/app/ml-shortcuts)— an app I developed that lets you control your Mac using the models you create with CreateML. Just import your model, map gestures to keyboard shortcuts, and you’re ready to go! It’s especially useful for designers working in tools like Figma, where you can create interactive prototypes controlled by real gestures (think next-level Wizard of Oz prototyping!).

[\[Download for free\]](https://www.annotate.supply/app/ml-shortcuts)

### **Other Things to Try**

* Add more gestures to recognize

* Try different hand poses or other classifier types (Object, sound, body, etc)

* Try adding negative or empty classifications (Not thumbs up)

* Create a simple app that uses your model

You’ve just created a machine learning model that can recognize hand gestures — and you didn’t write a single line of code! A few years ago, this would have required extensive knowledge of machine learning algorithms and mathematics. Today, you can do it in minutes with CreateML.

Pretty easy, wasn’t it? This is just the beginning of what’s possible when we combine machine learning with design and development. The ability to recognize gestures, poses, and movements opens up entirely new ways to think about user interactions and interfaces.

While tools like CreateML have made machine learning more accessible, I believe we can push this even further. Some parts of the process — like creating folder structures, gathering training data, and identifying false classifications — could be even more streamlined. That’s why I’m currently working on developing tools to make this process even more accessible for designers and developers.

Want to stay updated on these tools and learn more about making machine learning more accessible? Subscribe to my Medium profile! I’ll be sharing:

* More tutorials like this one

* Updates on new tools and workflows

* Tips for integrating ML into your design process

* Real-world examples and case studies

Create Your First Machine Learning Model – No Coding Required.

Product Designer

Best of Swiss Web

For [EAO 3D Configurator](https://eao.com/catalog/de)\
1x Gold, 1x Silver, 1x Bronze, 1x Master Nomination

For [Thommen Medical Webshop](https://shop.thommenmedical.com)\
1x Gold, 2x Silver, 1x Bronze

For [E-Rara](https://www.e-rara.ch) & [E-Manuscripta](https://www.e-manuscripta.ch)\
2x Gold, 1x Bronze

For [Compona Webshop](http://compona.ch)\
1x Gold, 1x Silver, 1x Bronze, 1x Master Nomination

For [ETH Art Guide](https://art.ethz.ch/en/homepage/)\
1x Silver

MACH Impact Award

For [Thommen Medical Webshop](https://shop.thommenmedical.com)

Best of Swiss Apps

For [eZug](http://ezug.ch) App\
1x Silver, 3x Bronze, 1x Master Nomination

Intelliact is Middleware company and technical partner of Eyekon AG. After completing multiple successful projects together, Intelliact decided to redesign their brand in 2018. \
\
The goal was to make their story as accessible as possible: Explaining what a middleware company can provide to optimise internal structures. This was achieved by breaking their story down to the simplest form. The result is an isometric style with a generative properties.

Intelliact Rebrand

Doppelmayr is a world leader and manufacturer of ropeway systems. \
\
For their digitisation strategy, the company decided to redesign and simplify their control systems by approaching various processes with a user-first perspective – By reducing complexity by unification. And insuring participant safety thru realtime data integrations, allowing for better decision making. \
\
The resulting system is called "Doppelmayr Connect", and was released in 2017.

Doppelmayr HMI Connect

Compona is an electronic components distributor, which had been printing 1'000+ page catalogs for 40 years, even tho having a digital store front for 10. In 2017 they decided to become fully digital with their sales channels. \
\
The result is an award-winning new website with an innovative tree-view navigation structure(BoSW Gold, Silver & Bronze, 2019).

Compona Webshop

CELOS is an app-based control environment for the manufacturing industry. MAPPSV is the hybrid-digital interface platform (HMI) for precise and efficient control of machining tools. Together these systems allow users to manage documents, visualise processes, maintain overview / connectivity of machine data and planning of production lines on all corporate levels – Thereby laying the foundation for a true digitised, paperless production. \
\
This platforms design and concepts were developed at Schindler Creations for DMG Mori

DMG Mori CELOS / MAPPS V

Badgey App

Capture, annotate and train CoreML models on your device. From Sound, motion & video. Embeddable & easy to use. \
\
To learn more or become a beta tester, [click here to contact me](http://mailto:hi@luthi-mar.co).

Annotate Audio

Use your own CoreML models to interact with your Mac through keyboard shortcuts.

ML Shortcuts

Design digital credential and government solutions. Led all design and user development of Procivis One, a comprehensive credential lifecycle management software. Develop solutions for Swiss government entities. Collaborate across all teams and verticals. Implemented design libraries and templating. Coordinated & supported  (re)brand and website initiatives.

Lead Product Designer

Developed concepts, conducted user research, and implemented assets for clients including EAO, Thommen Medical, and Compona. Specialized in user research, rapid iteration, and creative coding.

Interaction Designer & AD

Offers conception, development, and sparring services to clients ranging from startups to cultural organizations.

Designer & Consultant

Focused on investment goods and industrial machines. Integrated touch screens into CNC and roadway equipment. Developed concept for tablet controller with joysticks for pipe laying machines. Emphasized safety and long-term usability in industrial design.

Interaction Experience Designer

Graphic Designer

Design Intern

Created graphics and animations for SMEs and larger companies. Produced graphics for Orell Füssli Books and animations for store fronts.

Marco Luthi

2024