User Experience Review of Muse Personal Agent

The user shares their experiences with Muse, highlighting its speed and capabilities in travel planning, shopping, music event tracking, health analysis, and finance management. They note both strengths and areas for improvement, particularly in user interface design and task management.

added by @shivambharuka

I have been playing with @Muse since yesterday and I am honestly impressed. Probably the fastest personal agent I’ve used so far. Here are a bunch of things I tried:

1. Travel: I asked it to find flight tickets from SFO to Munich for a 10 day trip in December that makes the most of the holidays. It did the search fast. Then I asked it to monitor for price drops and it scheduled a task to check it every Monday.

2. Shopping: Tried multiple vendors here. Amazon worked all the way from signing into my account to adding the item to my cart and placing the order. But on a later task, it needed to sign in again despite having the credentials in its vault. Not sure why it didn’t reuse the login, but this could get annoying quickly on services that require 2FA. Wayfair blocked it twice with bot detection. Walmart let it add the item to the cart but crashed during sign-in (Instinct worked here without any issues).

3. Instagram: This one was very interesting. I gave it a video link and asked for a transcript. It couldn’t get the video content through its native Instagram integration, so it figured out how to download it through the browser and set up Whisper CLI to transcribe it. This took more than 10 minutes the first time but subsequent requests were very fast because it reused the same setup.

4. Music: I asked it to go through my Spotify playlists and find upcoming shows in SF from artists I listen to. It pointed to a Bruno Mars show in October and then found resale tickets on Ticketmaster. It was then able to setup a task to monitor for cheaper tickets.

5. Health: I connected Apple Health and asked it to analyze my runs for the past few months and make a training plan for the Golden Gate half marathon in November. It did the analysis, added the plan to my goals, and created a mini-dashboard for tracking progress.

6. Finance: This one’s right up my alley. I connected a brokerage and a bank account to see how it works. For simple tasks like finding subscriptions and analyzing cash flow, it did the job fine. For anything complex, it just said to consult a financial advisor (make sense - this is why @getfinnnow exists).

Few thoughts:

- It's incredibly FAST. With Instinct, my behavior has been to give a task and forget about it because I have no clue when it will get back. Also, having a visual interface of what it is doing is quite helpful to see it making progress on long-running tasks.

- The first set of connectors are good. Native Instagram and Messenger access is definitely an advantage but also other connectors like Spotify, Health, etc out of the box gives it a good mix of everyday apps. I also liked that you can choose exactly which actions it can take with each connector.

- Some of the UI elements are quite slick. The avatar animating while it works on a task is cool to see. Having all scheduled tasks and goals in one place is useful too. I personally like the idea of one main thread and side chats, though moving between them and bringing things back into the main conversation needs to be designed better.

- The ability to create dashboards (called artifacts) is promising. There have been a few attempts to create platforms where people can create mini-apps. Combining it with personal assistant makes a lot of sense since it solves the cold start problem of knowing what to build. However I do think the Muse’s version requires opinionated design work before being really useful.

- The overall app feels too cluttered. There’s chat with side chats, feed, ideas, goals, and artifacts. I like having a dedicated app rather than doing everything through iMessage. But I don’t think an everyday consumer needs all these separate sections, especially on day one. I would get rid of feed (or bring it together with main chat), put idea nudges inside goals, and tuck artifacts somewhere in the settings.

- I’m also a little unclear on who this is built for. Parts of it feel like a coding agent, and then there’s a whole menu of personal assistant tasks. There is a lot you can do with it but it doesn’t really guide you towards an experience that you would find value in. Right now it feels more suited to people who are interested in agents than the billions of users Meta could service.

- I also don’t think a consumer app needs to show every API call, terminal command, and file download like a coding agent does. Seeing high-level progress and steps is useful but these details are unnecessary. It might also make it more prone to attackers. During the transcription task, Muse printed a proxy URL with a username and an unmasked password/token in the execution trace. I haven’t verified whether it works or what access it gives, but that shouldn’t be showing up in a user-facing trace.

- One angle I am curious about: could Meta use its existing relationships through the ads business to get direct integrations with merchants like Wayfair and Walmart? It would be interesting to see that play instead of the agent having to constantly fight bot detectors.

The bigger thing I struggle with Muse or really any other personal AI assistant today is figuring out what I will keep using them for. I am trying them out of curiosity and forcing myself to delegate tasks but I haven’t found the one or two things they do where I’m like “Okay, this solves a big headache. I don't have to think about this task anymore. AND I’m willing to pay for it.”

Overall, really impressed with Muse. There’s a lot to like here. I worked at Meta before starting Finn, so I’m rooting for them. Congrats team!