What you need to be concerned with are “contamination” and “leakage”. Those two terms are definitely something to keep in mind in your quest for anonymity.
It’s simple: everything you do online can be traced back to you in some way if you do not take precautionary measures to avoid being discovered. Since we’re talking about anonymity, let’s start with the first thing you use to get online — your computer.
If anonymity is important to you, it’s probably best to avoid buying your computer using PayPal or or your credit card. Cash is always the best way. Of course, Bitcoin has its place, but using Bitcoin to buy a computer online doesn’t change the fact that you will need an address to ship it to. Your best bet is to simply pay cash in store.
The best (and safest) way to keep your identity private is to not give out any information at all. If there is nothing on your computer that can be linked to you, then you have considerably less to worry about. Invest in a specific computer that you will use online, encrypt your hard drive (and create a hidden volume), and only use it for what it was intended for.