I would highly recommend that you get a self-contained kit with a USB-based J-tag debugger, and powered through the USB as well. This way, you don't need to buy separate J-tag pods or power supplies.
I suspect that many people would recommend something like Arduino, especially that the new Arduino DUE board comes with the ARM Cortex-M3 processor. But if you really want to learn about the ARM processor (and you specifically mention ARM), I think that you need a proper debugger to see the disassebled code, play with the ARM registers, etc. Arduino does not provide this capability, because Arduino does not give you the built-in J-tag debugger.
So, here is my list of inexpensive self-contained boards:
1. Stellaris Launchpad from TI (http://www.ti.com/ww/en/launchpad/stellaris_head.html) $12.99, Cortex-M4F
2. LPCXpresso boards from NXP (http://ics.nxp.com/lpcxpresso/), around $30, Cortex-M0, Cortrex-M3
3. STM32 discovery from ST (http://www.st.com/internet/evalboard/product/252419.jsp) $14.99, Cortex-M4F
4. Freedom FRDM-KL25Z from Freescale (freescale.com/FRDM-KL25Z), Cortex-M0+
From these boards, my favorite is the Stellaris Launchpad with the latest Cortex-M4F CPU. In fact, I'm using this board in my YouTube video course "Embedded Systems Programming Course" available at
http://www.youtube.com/p...hwyTzI3BHwBLbGQoPFxPAPM