When I visited one spring weekend, the wildflowers were in full bloom and the weather was quite warm but not overly hot. Coastal breezes and fog keep the area pretty darn nice.
There are lots of good restaurants in downtown SLO,just walk down Monterey or Higuera and take your pick (Natural Cafe is an easy good salad kind of place, Firestone has good pizza, and Big Sky Cafe is renowned for excellent food - salads, local meats, vegan and vegetarian ) also small local wine tasting shops and artisinal beer.
The local Farmer's Market is much more than a row of casual produce stalls. Folks drive from two or three hours distance to take in this happening weekly event.
Avoid just stopping off in SLO during high college times. Local state university Cal Poly Graduation is late June, and other significant college sporting and college spirit events can affect availability. The rest of the times you'll generally find lots of motels and hotels. Petit Soleil is a nice small B&B, within walking distance of downtown. It's just next door to fun little Splash Cafe, with easy meals and very tasty breads (you can buy those to go too). There are also more budget oriented motels in the same stretch. My husband is partial to "La Cuesta" which includes breakfast. Across the street, a little more expensive but fun, is Apple Farm Inn. It's a kitschy 70's kind of place. I'm partial to the very chintzy furniture, big soft beds and switch on fireplaces there, but if you don't like Disneyland it might not be your thing!
The stretch of Monterey where you find the motels, as you're walking downtown, and before you cross under the railroad trestle also has lots of fun little antique stores.
Many people stay in SLO before or after a trip to Hearst Castle. I've been to the castle, enjoyed it once, but I wouldn't probably go again. I like historic buildings but this one is a little overwhelming. Still, if you have time you might enjoy it a lot.
