Back in high school I listened to a band called Animal Collective ( I listen to a lot of Dilla, jazz liberatorz, mad villain, all that stuff too) who focused a lot of sampling and using interesting sounds and making them into music. I watched a couple of live sets that they've done over the past couple of years and started looking at the things that they use for playing their songs (MPC's, samplers and all that stuff) and thought it was pretty dope. About two summers ago (Currently in third year uni), one of my good friends bought a roland sp555 and we started playing around with it during our summer break. Couple of weeks after i bought myself a midi controller (akai mpk mini) and downloaded ableton live to start making my own stuff. Even though I don't have much time to play around in ableton i still find it fun making my own shit every once in a while. The best part of having these things and knowing how to sample is how awesome the end product comes out as. If you have a sound you have in mind, it may come out completely different when actually trying to put it on a program or w/e your using.
Obviously if you have the money for an actual sampler, by all means go for it. There are still other options available to you if you don't want to spend $500+. You can buy a midi controller, plug it into your computer, and use it as a controller. MPK mini's go for less than $100 (you can probably buy used ones for cheaper on ebay and shit), it's really up to you. You still need ALOT of time to get to know your machine (whether it'd be sampler or midi controller) as you'll probably experience a lot of technical issues starting up, and the basics like cutting samples and adding effects.