Notes to myself and others on AI & Design
Who knows where this trail goes? ~ Photo taken in Austin, 2005. This photo, and all the words here, by me.
I was asked to share some thoughts on how AI and design intersect, especially for junior designers. I thought it'd be great to expand on some notes I’ve been taking and offer some thoughts and guidance on how our design work will evolve with AI and all these new tools. The way we adapt to these tools will shape our evolution, and honestly, a lot of the advice I've shared has been relevant for quite some time. While the core principles of design still hold, they've evolved over the years and will continue to do so at a speed we’re probably not quite ready for. These notes are long, sort of rambly, and loosely organized, but I hope that by sharing them, someone will find something interesting or helpful.
Develop both Taste + Logic
Everyone using AI will need logic and taste, but it will become essential for product leaders across crafts. In a world where everyone can build, the individuals who can align and synthesize their ideas into a cohesive point of view become increasingly important. But also, those who can think and work in well-designed systems become invaluable.
Taste alone, in my opinion, will not be enough.
Understand the end-to-end
Having an end-to-end view of what should be built, how it should be built, and why it should be built will be key. This is good advice for product builders in general, but I believe it becomes even more critical as the tools become increasingly powerful—especially the why. There will need to be a strong POV directing the work, especially in larger organizations and across well-adopted consumer and enterprise spaces. Without this, we’ll see a lot of fragmented experiences, and I would imagine a loss of trust and confidence in what is built.
Embrace an experimental mindset
To get that end-to-end view, you’ll want to embrace an experimental mindset. Prototype ideas, explore, and share with others. Now more than ever, we need to feel like it’s ok to make mistakes in the early part of the process. On the other hand, that last 20% becomes doubly important, so having an eye for detail and the ability to execute the last mile is key.
Look at AI as a creative partner
AI can be scary, for sure, but it can also be powerfully creative if you let it and remain mindful and present. Sometimes, it’s a good idea to limit the information you give or be purposefully unclear and see what happens. There is a lot of freedom in embracing the chaos, leaning into the emergent creativity AI can bring, and leaving room for these tools to go wild, which can yield unexpected and interesting results. In a way, I think that’s one of the most significant benefits of AI. At the same time, it should be approached with care: hold on to the reigns and be wary. AI can amplify and assist your creativity, but it should never replace it.
For a personal project, I briefly explained the theme switcher concept and let v0 create something. What it did was not exactly what I had in mind, but I was fairly vague and expressed that I wanted to explore ideas. It selected the icons (based, I think, on seasons) and the colors, as well as developing the UI. I had to iterate to clean it up a bit, but this is a direction I liked and something I hadn’t initially thought of.
Work on soft skills
This is evergreen advice, in my opinion, but continue to work on being a good collaborator and communicator. We still haven’t figured out how to do this stuff as a team. I think learning how to work with these new tools collaboratively will be a huge unlock.
Become a master of rationale
I touched on this before, but I genuinely believe that the ability to communicate your ideas, mainly the underlying reasons, will become increasingly important. You’ll need to be able to explain decisions to others (and in your prompts) in a way that builds confidence and trust. This will be needed for both co-workers and end users (through apps and assistants), as well as for the agents themselves. I think now this is all wrapped in prompt engineering, but to me, that feels like a stopgap. Makers will need to communicate their ideas more clearly in ways that work for both themselves and the agents they work with.
Understanding the semantics and best practices of code
Even if you aren’t a coder, getting a fundamental sense of how things work will help. We’re not quite there yet, but I imagine coding agents will enable much more direct manipulation of code. If you understand how it’s supposed to work and fit together, you’ll be able to utilize that knowledge to be more effective and delve into the details of what you’re building. The more you know, the easier and less risky this will be.
Understanding of the models and what they can do
And what they can’t do. And how best to work with them. Chop things up into smaller tasks, move away from the desire to one-shot everything, try to be more specific, explore their limits, and use side projects to experiment with them. We need to lean into playing with these tools, and I’ve not only had a lot of fun doing so, but I’ve also learned a lot. They evolve so rapidly that it’s helpful to get in and experiment.
Collaborate AI to Collaborate with AI
Sometimes, the most effective way to harness the power of AI is to ask it how best to work together, whether with the model you're using or with other models. You can let it guide you and pose questions. I've discovered that encouraging it to be inquisitive or to help shape the questions you have can lead to powerful and enlightening insights. Look at AI as a teammate —one that can be taught, influenced, spoken to, and encouraged. One model to explore and try, fairly new to me but mirrors a bit how I’ve been working, is the “Stingray” model from Board of Innovation, but there is still a lot to be explored when it comes to collaboration models with AI, especially within teams.
Practice resilience and adapt well to change
The pace and rate of change, in my opinion, are literally (really) impossible to keep up with. Designers, especially, need to be curious, resilient, and adaptable. Practice fundamentals, get your time management and communication skills right, become a facilitator, and learn as much as you can, but also? Play around, make a mess, and experiment.
Think in terms of apps + assistants
We’re moving away from products. The apps will be more focused, smaller, and more straightforward; the assistants will become more intelligent, autonomous, and capable of performing more complex tasks, such as directing other, increasingly specialized agents. Over time, they will also be able to handle more natural direction. Also, breaking our products down into smaller pieces of functionality (I’ve been thinking in terms of “blocks” a lot lately) should help agents and users target and focus on what is the next best or most relevant thing. I still think there will be a need for persistent views and app functionality, but I expect users (and agents) will want to customize and expect UIs to adapt and change to meet their needs.
Trust and security will likely be a key differentiator
Table stakes will be high. It’s hard to know how things will play out, but everyone knows AI can make mistakes. Sure, sometimes it’s a bit silly, like this AI-generated summary reading list with books that don’t exist, but when it comes to enterprise data? That’s a different story, and it doesn’t take much imagination to see how things can get worse from there. It’s always best practice to review the work, and I think humans will always be involved in some way with most of what we do; however, AI brings a significant amount of risk. We’ll need to consider both the perception of trust and the real-world risks associated with using and expressing AI in our work.
On direction and working with agents
With current best practices around prompt engineering. Learn the rules and leverage the tools. There are many tools, such as Anthropic’s Console, that can help improve your prompts. These aren’t particularly helpful without a fundamental understanding of how the models work and the various levers and inputs that can be adjusted. Understanding how factors such as temperature, roles, and rules work and how they impact your outputs is extremely helpful. But they are getting better. Tools like v0’s “enhance prompt” can be helpful, and I think they’ll improve. However, sometimes it’s best to communicate naturally, and I predict that’ll be the future of agent interaction. Almost all of the advice for using these tools begins with some variation of “start with clear instructions.”
With natural language. I believe natural language will become increasingly important, both for designers to communicate with and to consider in their work. I’ve heard the terms NLX and NLE for natural language experience, and I think there is something there. This is how we will interact with assistants. And, frankly, I believe that prompt engineering will become less important as the tools become smarter, and we’ll need more natural ways to interact. Even now, there are many tools, some built-in, that can help you engineer and improve your prompts. You still need to be clear and concise going in. Practice writing, improve your writing skills, and work on expressing your ideas as a foundational skill.
Tip: when in doubt, just ask. I’ve found that asking an agent to teach me, or tell me, what it needs has been surprisingly effective. I also do a fair amount of typing something and then asking the agent to make it helpful for itself, and to ask me questions if needed. I’ve got real concerns about the loss of knowledge in craft in light of AI, but there are ways these tools can be used to learn, and hone, craft and sometimes it’s as simple as asking them to teach.
Regarding the future of configuration and personalization
How will personalization work in the app and assistant world? Will the assistant handle this, or will there still be a need for user config? There will still be a need for persistent views, such as dashboards and UI-based interactions that are learned and mastered, but there will likely be more fluid, dynamically generated views. We’ll have configuration that goes beyond simple show/hide/config and more into rule sets and, maybe a ways into the future, the assistant adapting to the user via their usage, making suggestions and tweaks to the experience automatically (if the user wants that).
Regarding growth in a world of AI
A note to leaders and those who are working towards design leadership.
As you move forward (and up) in your career, it can become a little isolating, especially with the rise of remote working. Having solid partners--cross-functional partners, a good craft or IC partner--helps a lot, but it’s not always something you’re going to have. Scaling yourself and your skills is hard on so many levels. Finding time to improve your skills while also mentoring others is…haha…well, it feels impossible sometimes. AI can both help with this and add a tremendous amount of pressure. Things are changing, and we need to adapt. Our teams look to leaders for support and guidance. It’s one of the reasons I’m writing this right now. It feels both daunting and exciting at the same time, and at least in my own experience, AI has made my work more complicated when viewed holistically. However, I believe there is hope, and it starts with leadership, sharing, and exploring together, as well as bringing others along for the ride and working as a team. And while I’ve found that while an AI partner can’t replace another person, it can help in some meaningful ways. With the ever-growing mountain of tasks and things to learn, with organization and time management, but also as a way to bounce ideas around when nobody is around. I can’t say it’s helped at all with isolation, and there is a part of it that I think makes it feel a little worse, but in time, I think there is an opportunity for designers and the people we serve to free up valuable time to connect with others. I, for one, am looking for AI to eventually take some of the load I have so that I can spend more time with people.
Regarding the ethical concerns of using AI
I have a lot of concerns and more questions than it’s easy to hold in my head. To say I’m conflicted would be an understatement. I believe the tech industry has a significant amount of work to do in reducing the impact of AI on both people and the environment. But it seems clear that these tools are here to stay and will continue to evolve, which creates a gnarly dilemma. I think one thing we can all do is work with them mindfully and with restraint. I don’t personally think we should be asking AI technology to do everything we don’t want to do. I don’t think that’s a smart or sustainable approach. For one thing, it should not replace fundamentals: writers need to know how to write to get the most out of AI, for example. It shouldn’t replace writing for writers or art for artists. There is also the joy of learning and making things yourself. For me, this is a positive of these tools; I can now do things that I couldn’t before. At the same time, I don’t want to outsource my brain or creativity to them. And then there is the environmental impact. There is a lot of conflicting information about it, but it’s safe to say that it’s worth considering. We should think carefully about using AI for trivial tasks as much as possible. Yes, there is a nice promise in taking out the drudgery of work, but there are also costs associated with it. At least for me, I’m trying to reduce my usage down to things that have real value. Yes, I think playing with these tools is a good way to learn, but I want to put as much of my own effort into it as possible and not use them for entertainment, if that makes sense. Don’t use AI, collaborate with AI. I think we have some large, complicated questions to answer with this massive shift, and we’re just at the start. I hope design can contribute to that in positive and meaningful ways.
Regarding the speed and pressure of…everything
If all of this stresses you out and feels like a lot, know that you’re not alone. I feel it too. At times, it’s energizing, but just as often, it’s draining. Take it slow, take a nap, take a walk…take time for whatever you need.
