You're not doing anything wrong at all!!!
Some of us use a technique commonly referred to as "parking." Basically, when you are done with a color in an area, you carry it over to where you need it next and literally park the thread and needle. That way, it's ready to go when you are ready for it!
My experience with this technique has taught me that when I do this I start and stop threads less often, my back is sometimes neater, it saves me some time, and if I messed up and the thread I used is still "live," it's often easier to fix (I hate picking out my knots). But, I do have to double check where I park things to a) make sure it's in the right place and b) it's the color/symbol I think it is. I've gotten into the habit of putting all of my bobbins for the colors that I have parked in a separate spot in my thread caddy (or I grab an spare container of whatever size I need) so that it's easier to figure out what color/symbol it is. And, there will be times when I have to just sit with my project and untangle it a bit
Whether or not you park your threads is totally up to you and what works best for you! (example, I park my threads constantly, my mom (who taught me to stitch and has stitched for YEARS) doesn't (actually, to quote her, she "does not, cannot, will not"
Hope this helps!
Can't wait to see your progress!