Kevin Neely's Security Notes

chatgpt

Creating a Michelin menu with Generative AI

We've been watching season three of The Bear and it is totally nuts, filled with drama and psychology, and human relationships, and –when they can get around to it– cooking food. This is a fantastic series, and as foodies, you can imagine that my wife and I really enjoy it. The series is intense, following a classically-trained, at the top of his game chef as he “drops down” from a fancy restaurant to a steak sandwich shop and works back toward where he was before.

That’s not a spoiler, it’s the premise. And this isn’t a review, it was just something in my mind as I experimented with this new tool called LobeChat, which is an interface to #genAI inference servers such as #OpenAI’s #chatGPT, #Claude, and nearly any other commercial or #opensource system you can imagine. I'm tweaking a #homelab hosted genAI inference server and this post is about experimenting with [Ollama]() and a Lobe chat front end for easy access. I haven’t formed a final opinion on this setup at this time, but the #LobeChat front end allows for the quick switching between inference servers and their hosted models. While not being super easy to configure, once it works, the whole thing looks very easy for someone, i.e. a roommate or a spouse, to flip between models -even in the same conversation.

So let’s look at some results. Of course, the results are about the models themselves, but the point here is that LobeChat made it easy to switch between them as I continued to develop the conversation.

First prompt

This was with ChatGPT (4o, I think). I needed something to test that an integration was working. We had just watched The Bear so this is what I came up with:

as fast as you can, what's the best restaurant dinner menu with wine pairing that could win a Michelin 3 star award?

Response

Crafting a Michelin 3-star worthy dinner menu with wine pairings involves high-quality ingredients, exquisite preparation, and a thoughtful combination of flavors. Here’s a conceptual menu that could impress:

Amuse-Bouche:Oyster with Champagne FoamWine Pairing: Champagne Brut

Appetizer:Seared Scallops with Cauliflower Puree and CaviarWine Pairing: Chablis, Premier Cru

Soup:Cream of White Asparagus with Truffle OilWine Pairing: Viognier

Fish Course:Poached Turbot with Fennel and Citrus Beurre BlancWine Pairing: Pouilly-Fumé

Main Course:Roasted Duck Breast with Plum Sauce, Parsnip Mash, and Seasonal VegetablesWine Pairing: Pinot Noir, preferably from Burgundy

Cheese Course:A Selection of Artisan Cheeses with HoneycombWine Pairing: Vintage Port

Dessert:Chocolate Lava Cake with Raspberry Coulis and Homemade Vanilla Bean Ice CreamWine Pairing: Recioto della Valpolicella

Mignardises:Petit Fours and Truffles

This menu balances rich flavors with delicate preparation, offering a progression of dishes that showcase both technique and creativity. Each wine pairing is chosen to enhance the flavors of the dishes, aiming to create a harmonious dining experience that could captivate both critics and diners alike.

Review

Not a bad first response, especially since I was just checking to see if the system worked.

Prompt 2: Get ChatGPT to refine it

That was pretty boring. I’m thinking early 90s fancy dining at best, but I wasn’t eating fancy back then and assume it was all shoulder pads, tuna steaks, and cocaine. Continuing with ChatGPT, I posed the following in an attempt to refine our menu (in retrospect, I was generous with the stars):

That's an incredibly boring menu. At best 2 stars. And that's in a region that is not known for its gastronomical excellence

Response

Creating a truly innovative and Michelin 3-star worthy menu certainly demands a more avant-garde approach, pushing culinary boundaries with unique combinations and exceptional techniques. Let's elevate the original concept to something more daring and creative:

Amuse-Bouche:Nitro-frozen Lime and Ginger Pearls with Micro BasilWine Pairing: Small-batch Sake

First Course:King Crab Tartare with Avocado Mousse, Mango Gel, and Edible FlowersWine Pairing: Albariño

Second Course:Foie Gras Au Torchon with Saffron Pear Compote and Brioche CrispWine Pairing: Sauternes

Fish Course:Butter-Poached Monkfish with Leek Ash, Sea Beans, and a Lemon Verbena EmulsionWine Pairing: White Burgundy

Intermezzo:Cucumber and Elderflower Sorbet

Main Course:Sous-vide Venison Loin with Black Garlic Purée, Wild Mushrooms, and a Juniper Berry ReductionWine Pairing: Barolo

Cheese Course:A Selection of Aged Cheeses with a Fig and Walnut TapenadeWine Pairing: Aged Madeira

Dessert:Deconstructed Passion Fruit Tart with Coconut Foam, Crystallized Ginger, and Meringue ShardsWine Pairing: Late Harvest Riesling

Petit Fours:Matcha Green Tea Macarons, Raspberry Pâte de Fruit

This revised menu introduces more complex techniques and unusual ingredients, aiming to surprise and delight with every course. Wine pairings are carefully selected to complement and enhance these adventurous flavors, providing a sophisticated and memorable dining experience.

Review

Not bad! It would be nice if the wines were more than broad regions, however, I do appreciate the ‘ñ’ in Albariño. There’s nothing particularly innovative here; the fancy-sounding “Foie Gras Au Torchon with Saffron Pear Compote and Brioche Crisp” is a Thomas Keller recipe, so at least we’re pulling from the top now that I asked it to up its game.

Prompt 3: Let's try a different model

Not only am I using a different model at this point, llama3:instruct, but I am going to specify a food critic in order to change the menu.

Prompt

One more try: let's make a menu that would knock even James Beard out of his chair because it's so good

Response

You are absolutely right! That menu, while technically sound, lacks the daring, innovation, and personal touch needed to truly impress Michelin inspectors and earn those coveted three stars. It plays it too safe, relying on classic pairings and predictable dishes.

To push towards that 3-star level, we need to think outside the box:

  • Ingredient Focus: Instead of “seasonal vegetables,” source hyper-local, unique produce with a story. Maybe a heirloom tomato variety grown in volcanic soil, or foraged mushrooms with a distinct flavor profile.
  • Deconstruction & Fusion: Take classic elements and reimagine them. A deconstructed apple pie might involve caramelized apple spheres, crumble as a powder, and a cider foam. Fusion could bring Japanese umami to the duck dish, or Indian spices to the scallops.
  • Unexpected Textures: Play with contrasts – crunchy, creamy, smooth, airy. Think smoked paprika meringue alongside the chocolate lava cake, or a black truffle “soil” under the turbot.

Wine Pairings Should Be Equally Bold: * Instead of a basic Chablis, consider an obscure producer from the Côte de Beaune known for mineral-driven wines.
* The Pinot Noir could be a lesser-known region with unique terroir, like Slovenia's Primorska. * Experiment with natural wines, or even sake pairings that complement specific dishes.

The Presentation Matters:

Plating should be artistic, telling a story through the arrangement of colors, textures, and shapes. It’s about creating an immersive experience for the diner.

A Michelin-worthy menu isn't just about taste; it's about pushing boundaries, showcasing creativity, and offering a truly unforgettable journey through flavor.

Review

This is less of a menu than some advice for the aspiring chef. Let’s try something different.

Prompt 4: Let's go for Three Stars!

I felt like this was the time to go for the gusto, let's see how a different model would respond, but also with all the history behind it.

Prompt

I asked: > One more try: let's make a menu that would knock even James Beard out of his chair because it's so good

While it's not a crazily-innovative menu, it has some classic touches of a what an American, Michelin-starred menu would look like, and

Response with gemma2

You want to blow James Beard away? This isn't about ingredients, it's about experience.

The Menu: A Symphony in Five Acts

  • Act I: Awakening – Smoked Salmon “Caviar” with Beetroot Foam, Rye Crisp, and a single drop of Ossetra Caviar.

    • Pairing: Dry Riesling from Germany's Mosel Valley, expressing its slate minerality alongside the salmon's richness.
  • Act II: Earth & Sky – Roasted Quail “Confit” with Black Truffle Jus, Heirloom Carrot “Clouds,” and a dusting of Micro Thyme.

    • Pairing: Oregon Pinot Noir, showcasing earthy notes that harmonize with the truffle while its bright acidity cuts through the richness.
  • Act III: Fire's Dance – Charred Octopus with Smoked Paprika Aioli, Caramelized Onion Jam, and a drizzle of Chili Infused Olive Oil.

    • Pairing: Spanish Albariño, crisp and mineral-driven, its citrus notes complementing the octopus while balancing the smoky paprika.
  • Act IV: Ocean's Bounty – Pan-Seared Sea Scallops with Saffron Risotto “Foam,” Dulse Crisp, and a single Pearl Oyster.

    • Pairing: Chablis Premier Cru, its intense minerality mirroring the scallops' brininess, while subtle citrus notes elevate the saffron risotto.
  • Act V: Sweet Serenade – Deconstructed Peach Cobbler with Lavender Honey Ice Cream, Toasted Almond Brittle, and a hint of Rosemary Extract.

    • Pairing: Late Harvest Gewürztraminer from Alsace, its floral aroma echoing the lavender, while its sweetness complements the peach cobbler's richness.

The Experience:

  • Each course is plated as a miniature work of art, using contrasting colors, textures, and temperatures.
  • The dining room itself is a sensory experience, with soft lighting, calming music, and subtle aromas that enhance each dish.
  • Service is impeccable, with knowledgeable staff guiding guests through the tasting menu and explaining the inspiration behind each course.

This is not just food; it's a journey for the senses, a story told through flavor and texture, leaving James Beard speechless (and perhaps, slightly envious).

Review

Wow, what a different response! This one could actually be a menu, with the wines having robust and specific descriptions. I especially like how the wines are sourced from all over the globe, really picking best of breed in the selections. As good as the wines are, however, the food does not look 3-star level to me. The dishes sound really nice, and there’s a fair bit of molecular gastronomy in there. However, aside from the poetic nature of the menu, I’m not sure this is a top-tier menu for a chef to execute.

Still, most of these items are good enough for a fake restaurant. If this was to show up in a video game or a novel, I think most people would accept it and move on.

TL;DR

For the impatient, here is the final output from using ChatGPT to create service description documentation: – Final doc in Markdown format – (GitHub Gist) – Final doc in PDF format – (Box)

Overview

I’ve been working on building out an internal offensive security function and got to the point where I need some internal documentation as to the service(s) description, engagement model, outcomes, etc. Like a lot of planning, I started with an Xmind #MindMap, and with all the buzz around #ChatGPT, I wanted to see how well it could take what I have and build some docs for me.

In addition to Xmind, I use Obsidian for note-taking, jotting down thoughts, and organizing documentation. The object of this exercise will be to have #Markdown formatted text that I can make final edits in Obsidian and from there publish to a documentation repository.

ChatGPT Prompts

First prompt

I started with a pretty robust prompt. With the exception of the Objective paragraph at the top, this was 100% copy-paste from Xmind to the ChatGPT prompt.

Objective: I want to create a service description and engagement model for an internal red team. This service description should be formatted like a document with section headings and subheadings. Format the output in Markdown. The service components will be as follows: - the team will manage the scheduling process - the team will determine the activities that must be completed to adequately test the target - the team will determine whether a specific test should be carried out by an internal team or if an external testing firm needs to be engaged Pre-requisites for initiating a test include: - a system architecture diagram - a completed threat model document - access to the systems that will be tested The activities performed by this function will include the following items Document processes so they are Repeatable Defendable Create and perform Reporting with the following characteristics templates regular cadence tied-in to the risk review process findings go into backlogs, jiras, and project plans, depending upon the team and target demonstrated ability to remediate findings 

Engagement with developers must include 
  Regular sessions with R&D and IT teams that are developing tools and systems  
  Demonstrations 
  Threat modeling 
  
Engagement with System Operators must include 
  Educational review sessions with teams 
    system operators 
    Blue team 
    system integrators Demonstrations of exploit Collaborative remediation sessions Threat modeling

and the output ChatGPT output 1

Second prompt – measurable outcomes

Now I need to define some measurable objectives of the program in #OKR format so we can measure its success. The results were pretty meh. I’ve tried OKRs and #infosec success criteria in other chats and they pretty much all come out the same. This section had to be heavily edits foor the final. Luckily, I’d already outlined them in the mindmap.

ChatGPT OKR prompt and response

Prompts 3 thru 5 – flowchart

Now that I have a working service description and some needs-editing OKR (all a bunch of words no one will read), I need a flowchart diagram. I’m a fan of mermaid diagrams for their usefulness across various markup languages like Markdown and RestructuredText.

prompt

based upon the service description above, create a flowchart of the entire process beginning with scheduling and ending with concluding a set of tests. Output this flowchart in Mermaid syntax.

output

graph TD
A(Schedule Test)
B(Review Prerequisites)
C(Plan Activities)
D(Determine Testing Resources)
E(Perform Testing)
F(Record Findings)
G(Produce Report)
H(Engage with Developers and System Operators)
I(Conclude Tests)

A --> B
B --> C
C --> D
D --> E
E --> F
F --> G
G --> H
H --> E
H --> I


In this flowchart, the process begins with scheduling a test (Step A), followed by reviewing the prerequisites for the test (Step B). The team then plans the activities required for the test (Step C) and determines whether internal or external testing resources are required (Step D). The testing is then performed (Step E) and the findings are recorded (Step F). The team then produces a report (Step G) and engages with the developers and system operators (Step H) to remediate any issues. The process concludes with the finalization of the tests (Step I).

This was as decent start, and I had a couple iterations:

  1. adding a decision add a decision to the chart after step E in order to determine whether the completed tests are adequate to move on to testing completion
  2. inserting some more steps In the Mermaid flowchart above, tests need to be concluded prior to producing a report. Also, add a step called "Send to Product Management and Security Leadership" occuring in parallel with the "Engage with Developers and System Operators". Also, conclude with a step called "apply recommendations to target systems"

There were still some errors at this point, such as ChatGPT insisting on putting the “Conclude Tests” step at or very near the end, even though it’s really about the mid-point of the workflow. I could have continued the prompts, but at this point I decided to finish this off by hand.

Prompts 6 and 7 – RACI

Now that the processes are defined, we need to identify the responsibilities of each of the roles and how they interact.

ChatGPT prompt requesting a RACI for an offensive security program

From this, it’s clear that ChatGPT is good at formatting and generating a RACI matrix in #Markdown format, and this version has reasonable values for the roles, even if I don’t agree with them. Still, it was useful enough to have a good foundation that I could tweak, but before that, I added one last parameter:

add a new role to the matrix called Red Team Manager. This role should be responsible for teh scheduling and accountable for everything else. Add two new processes called Review Findings and Remediate Findings. The Developer is responsible for the former and the System Owner is responsible for the latter. Recreate the RACI matrix with these new parameters and output the Markdown code.

And this changed the RACI to basically make the manager accountable for everything.

Finishing up

At this point, I felt like I had the elements I needed, so I began the process of copy-pasting them from the interface into Obsidian and making tweaks to get a usable service description document.

The final output from using ChatGPT to create service description documentation: – Final doc in Markdown format – (GitHub Gist) – Final doc in PDF format – (Box)