I'm very late to the party I know but recently i've been getting addicted to podcasts. I guess its taken a while for there to be any decent ones that relate to the stuff I'm into. There are two i'm going to pick out for your attention.
Firstly Matt Barr's Looking Sideways which is a a couple of years old now. Matt has a long history in "action sports" media from editing whitelines magazine years ago to ACM media now which creates much of the action sports content you read in the regular papers as well as helping to curate the image of brands like Finisterre. The podcast is a regular series of interviews with a selection of characters from outside the mainstream covering everything from surfing to skating, skiing, climbing, snowboarding and even urban running. Matt is great at getting his guests off the track of other interviews you may have read with them and even if you don't participate in the sport the person is from, Matts interview style and content always makes them a good listen.
Second mention is for the Surf Splendor network which is an expanding group of shows that are all surfing based. Regular pods cover design interviews with shapers, the latest competition wsl news and the superbly salacious "grit" gossip show with Chas Smith. Theres a lot of great stuff in their archive, particularly a couple of shows with Devon Howard talking Skip Frye gliders and his signature Egg design.