AI for Craftsman Projects: Generating CNC Plans from Voice Descriptions

I was standing in my shop last November, staring at a piece of walnut I'd been saving for two years, trying to figure out how to turn a vague idea in my head into something my CNC router could actually cut. The idea was a topographic relief map of Denali — something I wanted to hang above the fireplace. I could see it perfectly in my mind. I had no idea how to translate that mental image into G-code.

That's the gap most craftspeople hit. The creative vision isn't the hard part. The CAD-to-CAM-to-machine pipeline is the hard part. You know what you want to build, but the software between your brain and the spindle motor feels like it was designed by people who've never actually gotten sawdust in their lungs.

So I built a system that lets me describe what I want to make, out loud, and get back a parametric design I can feed into my CNC workflow. It's not magic. It's duct tape and AI. But it works, and it's changed how I approach projects in the shop.

---

The Problem with Traditional CNC Workflows

If you've never worked with CNC, here's the typical workflow for a custom project:

1. Sketch the idea on paper or in your head
2. Open CAD software (Fusion 360, SolidWorks, FreeCAD)
3. Spend three hours trying to remember how parametric modeling works
4. Create a 2D or 3D model
5. Switch to CAM software to generate toolpaths
6. Configure feeds, speeds, bit sizes, depths of cut
7. Post-process into G-code for your specific machine
8. Run the job and hope you didn't mess up step 6

Steps 2 through 4 are where most hobbyist woodworkers and craftspeople bail out. CAD software has a brutal learning curve. Even Fusion 360, which is probably the most accessible option, requires significant time investment before you can model anything beyond a rectangle with rounded corners.

I've been using CNC machines for about eight years now. I have a Shapeoko XXL in my shop and I've built a few projects on larger industrial machines when I had access. And I still spend more time fighting CAD software than I spend actually cutting material. That ratio has always bothered me.

---

What If You Could Just Describe What You Want?

The idea hit me while I was using Claude to help refactor some code for Grizzly Peak Software. I was literally dictating requirements into a voice memo and then feeding them to Claude. And I thought: why can't I do this for physical objects?

The core concept is simple:

1. Speak a description of what you want to make
2. Transcribe the speech to text (Whisper)
3. Send the text to an LLM to generate a parametric design description
4. Convert that description into a format your CAD/CAM software can import
5. Review, adjust, and cut

Each step has real engineering challenges, but none of them are unsolvable. Let me walk through how I built this.

---

Step 1: Voice Capture and Transcription

I use OpenAI's Whisper for speech-to-text. You can run it locally or use the API. For shop use, I run it locally because my shop doesn't have great WiFi and I don't want to depend on cloud connectivity while I'm covered in sawdust.

var fs = require("fs");
var path = require("path");
var { execSync } = require("child_process");

function transcribeAudio(audioFilePath) {
    // Using whisper.cpp for local transcription
    var whisperPath = path.join(__dirname, "whisper.cpp", "main");
    var modelPath = path.join(__dirname, "whisper.cpp", "models", "ggml-base.en.bin");
    var outputPath = audioFilePath.replace(path.extname(audioFilePath), "");

    var command = whisperPath +
        " -m " + modelPath +
        " -f " + audioFilePath +
        " -otxt" +
        " -of " + outputPath;

    try {
        execSync(command, { timeout: 60000 });
        var transcript = fs.readFileSync(outputPath + ".txt", "utf8").trim();
        console.log("[transcribe] Got " + transcript.split(" ").length + " words");
        return transcript;
    } catch (err) {
        console.error("[transcribe] Failed:", err.message);
        return null;
    }
}

For recording, I just use my phone's voice memo app and transfer the file. Nothing complicated. The key is being specific when you describe the project. Here's an example of a good voice description:

"I want to cut a serving board from a piece of maple that's 18 inches long by 8 inches wide by three-quarters of an inch thick. The board should have a handle on one end with a thumb hole, rounded edges on all sides with a quarter-inch radius, and a shallow juice groove around the perimeter about half an inch from the edge and an eighth of an inch deep. The handle should be about 5 inches long and taper from the full width down to 3 inches."

That's specific enough for an LLM to work with. Vague descriptions like "make me a cutting board" will get you vague results.

---

Step 2: LLM-Powered Design Generation

This is where the real magic happens. The LLM takes your natural language description and converts it into a structured parametric design. I use Claude for this because it handles spatial reasoning better than most models I've tested, but GPT-4 works well too.

The system prompt is critical. You need to teach the model what CNC-relevant parameters matter:

var DESIGN_SYSTEM_PROMPT = [
    "You are a CNC machining design assistant for woodworking projects.",
    "When given a project description, generate a parametric design specification.",
    "",
    "Your output must be valid JSON with the following structure:",
    "{",
    '  "project_name": "string",',
    '  "material": { "type": "string", "length_inches": number, "width_inches": number, "thickness_inches": number },',
    '  "operations": [',
    "    {",
    '      "name": "string",',
    '      "type": "profile_cut|pocket|drill|engrave|contour",',
    '      "tool": { "type": "end_mill|ball_nose|v_bit", "diameter_inches": number },',
    '      "depth_inches": number,',
    '      "geometry": { ... }',
    "    }",
    "  ],",
    '  "suggested_feeds": { "feed_rate_ipm": number, "plunge_rate_ipm": number, "spindle_rpm": number },',
    '  "notes": ["string"]',
    "}",
    "",
    "Geometry can be described using primitives:",
    "- rectangle: { x, y, width, height, corner_radius }",
    "- circle: { cx, cy, radius }",
    "- ellipse: { cx, cy, rx, ry }",
    "- path: { points: [[x,y], ...], closed: boolean }",
    "- groove: { offset_from_edge, width, depth }",
    "",
    "Use reasonable defaults for feeds and speeds based on the material type.",
    "Assume a hobby-grade CNC router (Shapeoko, X-Carve, or similar).",
    "Always include safety notes about workholding and bit selection."
].join("\n");

function generateDesign(transcript) {
    return callClaude(transcript, {
        systemPrompt: DESIGN_SYSTEM_PROMPT,
        model: "claude-sonnet-4-20250514",
        maxTokens: 4096,
        temperature: 0.3
    }).then(function(result) {
        // Extract JSON from response
        var jsonMatch = result.text.match(/\{[\s\S]*\}/);
        if (!jsonMatch) {
            throw new Error("No JSON found in design response");
        }

        var design = JSON.parse(jsonMatch[0]);
        console.log("[design] Generated " + design.operations.length + " operations for: " + design.project_name);
        return design;
    });
}

For the serving board description above, Claude generates something like this:

{
    "project_name": "Maple Serving Board with Handle",
    "material": {
        "type": "hard_maple",
        "length_inches": 18,
        "width_inches": 8,
        "thickness_inches": 0.75
    },
    "operations": [
        {
            "name": "Juice Groove",
            "type": "pocket",
            "tool": { "type": "ball_nose", "diameter_inches": 0.25 },
            "depth_inches": 0.125,
            "geometry": {
                "type": "groove",
                "offset_from_edge": 0.5,
                "width": 0.25,
                "region": "main_board"
            }
        },
        {
            "name": "Handle Thumb Hole",
            "type": "profile_cut",
            "tool": { "type": "end_mill", "diameter_inches": 0.25 },
            "depth_inches": 0.75,
            "geometry": {
                "type": "ellipse",
                "cx": 15.5,
                "cy": 4,
                "rx": 0.75,
                "ry": 0.5
            }
        },
        {
            "name": "Board Profile with Handle Taper",
            "type": "contour",
            "tool": { "type": "end_mill", "diameter_inches": 0.25 },
            "depth_inches": 0.75,
            "geometry": {
                "type": "path",
                "points": [
                    [0, 0.25], [13, 0.25], [18, 2.5],
                    [18, 5.5], [13, 7.75], [0, 7.75]
                ],
                "closed": true,
                "corner_radius": 0.25
            }
        },
        {
            "name": "Edge Roundover",
            "type": "contour",
            "tool": { "type": "ball_nose", "diameter_inches": 0.25 },
            "depth_inches": 0.125,
            "geometry": { "type": "follow_profile", "offset": 0 }
        }
    ],
    "suggested_feeds": {
        "feed_rate_ipm": 60,
        "plunge_rate_ipm": 15,
        "spindle_rpm": 18000
    },
    "notes": [
        "Use double-sided tape and sacrificial board for workholding",
        "Cut the profile last to maintain workholding stability",
        "Hard maple is tough on bits — use sharp carbide end mills",
        "Consider a finishing pass at reduced stepover for smooth edges"
    ]
}

Those notes at the end? That's one of the things I love about using an LLM for this. It knows about material properties and machining best practices. A traditional CAD tool gives you geometry. This gives you geometry plus the practical knowledge of someone who's actually cut hard maple before.

---

Step 3: Converting to SVG for CAM Import

Most CAM software can import SVG files. So I convert the parametric design into SVG paths that can be brought into Carbide Create, Easel, Fusion 360, or whatever you're using:

function designToSVG(design) {
    var material = design.material;
    var scale = 72;  // 72 pixels per inch for SVG
    var width = material.width_inches * scale;
    var height = material.length_inches * scale;

    var svg = [
        '<?xml version="1.0" encoding="UTF-8"?>',
        '<svg xmlns="http://www.w3.org/2000/svg"',
        '  width="' + width + '" height="' + height + '"',
        '  viewBox="0 0 ' + material.width_inches + ' ' + material.length_inches + '">',
        '',
        '  <!-- Material boundary -->',
        '  <rect x="0" y="0" width="' + material.width_inches + '"',
        '    height="' + material.length_inches + '"',
        '    fill="none" stroke="#999" stroke-width="0.02" />'
    ];

    design.operations.forEach(function(op, index) {
        var color = getOperationColor(op.type);
        var label = op.name.replace(/[^a-zA-Z0-9]/g, "_");

        svg.push('');
        svg.push('  <!-- ' + op.name + ' (depth: ' + op.depth_inches + '") -->');
        svg.push('  <g id="op_' + index + '_' + label + '">');

        var geo = op.geometry;

        if (geo.type === "circle") {
            svg.push('    <circle cx="' + geo.cx + '" cy="' + geo.cy +
                '" r="' + geo.radius + '" fill="none" stroke="' + color +
                '" stroke-width="0.02" />');
        } else if (geo.type === "ellipse") {
            svg.push('    <ellipse cx="' + geo.cx + '" cy="' + geo.cy +
                '" rx="' + geo.rx + '" ry="' + geo.ry +
                '" fill="none" stroke="' + color + '" stroke-width="0.02" />');
        } else if (geo.type === "path") {
            var d = "M " + geo.points.map(function(p) { return p[0] + " " + p[1]; }).join(" L ");
            if (geo.closed) d += " Z";
            svg.push('    <path d="' + d + '" fill="none" stroke="' + color +
                '" stroke-width="0.02" />');
        } else if (geo.type === "groove") {
            var offset = geo.offset_from_edge;
            var w = material.width_inches;
            var h = material.length_inches;
            svg.push('    <rect x="' + offset + '" y="' + offset +
                '" width="' + (w - 2 * offset) + '" height="' + (h - 2 * offset) +
                '" rx="' + (offset / 2) + '" fill="none" stroke="' + color +
                '" stroke-width="' + (geo.width || 0.25) + '" />');
        }

        svg.push('  </g>');
    });

    svg.push('</svg>');
    return svg.join('\n');
}

function getOperationColor(opType) {
    var colors = {
        "profile_cut": "#ff0000",
        "pocket": "#0000ff",
        "drill": "#00ff00",
        "engrave": "#ff00ff",
        "contour": "#ff8800"
    };
    return colors[opType] || "#000000";
}

Each operation gets its own SVG group with a distinct color, so when you import it into your CAM software, you can assign different toolpaths to different operations. Red for profile cuts, blue for pockets, orange for contours. It's a simple convention but it makes the CAM setup so much faster.

---

Real Projects I've Built with This System

This isn't theoretical. I've actually used this workflow for several projects in my shop.

The Denali Relief Map

The project that started it all. I described a topographic relief map of Denali, 24 inches by 16 inches, carved from laminated layers of birch plywood. The AI generated a series of pocket operations at increasing depths to create the topographic contour effect. I had to manually adjust the elevation data — that's one area where the AI doesn't have enough spatial precision yet — but the basic structure and toolpath strategy were solid.

The finished piece hangs above my fireplace. Every visitor asks about it.

Custom Dovetail Templates

I described a set of dovetail templates for hand-cut joinery — different pin spacings for different board thicknesses. The AI generated the geometry perfectly because dovetail dimensions are well-documented in woodworking literature, and LLMs have ingested most of that literature. I cut them from quarter-inch acrylic and they've replaced my commercial templates entirely.

Cribbage Boards

This one surprised me with how well it worked. I described a cribbage board with specific hole patterns, track layouts, and decorative elements. The AI knew the standard cribbage board dimensions (120 holes in tracks of 5, specific spacing) without me having to specify them. It even suggested appropriate drill bit sizes for standard pegs.

Sign Carving

I've made several carved wooden signs using voice descriptions. "A shop sign that says LARSON WOODWORKS in a serif font, 30 inches wide, v-carved into cedar, with a bear silhouette on the left side." The AI generates the text layout, suggests v-bit angles, and creates the geometry. I still have to source or create the bear silhouette as an SVG import, but the text layout and spacing saves me an hour of fiddling in CAD.

---

Where This Falls Apart (Honest Assessment)

Let me be real about the limitations, because I'm not trying to sell you something here.

Complex 3D geometry is still hard. The system works well for 2.5D operations — things where you're cutting profiles, pockets, and grooves at various depths. True 3D sculpting with ball-nose toolpaths is beyond what this text-based approach can handle well. For that, you still need proper 3D modeling software.

Dimensional precision requires iteration. The AI's first pass at dimensions is usually in the right ballpark but rarely perfect. A handle that should taper from 4 inches to 2.5 inches might come back as 4 to 3 inches. You have to review and adjust. I've learned to treat the AI output as a starting point, not a finished design.

Material-specific knowledge is uneven. The AI knows a lot about common hardwoods but less about exotic materials, composites, or metals. Feeds and speeds for hard maple are usually reasonable. Feeds and speeds for purpleheart or G10 are often wrong.

It doesn't replace CAM. This system generates geometry, not G-code. You still need CAM software to create actual toolpaths with proper stepover, stepdown, ramping, and lead-in/lead-out moves. The AI gives you a head start on the design side, but the machining side still requires human expertise and proper CAM tools.

---

The Intersection of Old Skills and New Tools

Here's what I find fascinating about this project. I'm a software engineer by trade, but woodworking is how I decompress. My shop is where I go to get away from screens. And now I'm bringing AI into that space.

But it doesn't feel like a contradiction. It feels like what technology is supposed to do — remove friction from the tedious parts so you can spend more time on the parts that matter. I don't enjoy fighting with Fusion 360's sketch constraints. I do enjoy choosing wood, cutting joints, and applying finish. The AI handles the part I dislike so I can get to the part I love faster.

There's a broader lesson here for anyone building AI tools. The best applications aren't the ones that replace human skill. They're the ones that bridge the gap between what someone can imagine and what they can execute. A master furniture maker doesn't need my system — they can model anything in their head and translate it to wood through decades of practice. But a hobbyist with a CNC machine and a great idea? They need a bridge. That's what this is.

I've been thinking about expanding this into a proper tool — something with a simple web interface where you can record a description, see the generated design, adjust parameters with sliders, and export directly to G-code for common machines. If that interests you, keep an eye on Grizzly Peak Software. It might become a real product. Or it might stay a hacky script I run in my shop. Either way, I'm having fun building it, and that's enough.

---

Shane Larson is a software engineer and the founder of Grizzly Peak Software. He builds software and furniture from a cabin in Caswell Lakes, Alaska, where the sawdust pile competes with the snow pile for most of the year. His book on training large language models is available on Amazon.