Avila Beach is very easy to get to. About 8 miles south of San Luis Obispo, you'll see signs to both Avila Beach and Port San Luis. It's a short but pretty drive.
When you arrive at the ocean, you'll see the town of Avila Beach off to your left. Stay on the road and you'll end up at Port San Luis Pier, now known as Harford Pier.
Maps