I want to collect oddities?

I recently started watching the show obscura and took an interest in the strange collections people had. I was wondering if anyone had a clue where to start an oddities collection?