Will it show a Linux option at the start time ?
It depends on your BIOS settings. It might boot off the external disk by default, otherwise you'd have to go into the BIOS settings and change the boot order. It depends on your specific BIOS how to do this, but it's generally pretty self-evident.
Will it only boot on the PC through which I have installed Linux on external Harddisk ? If yes, hiw can I make it to boot from other PC's also ?
By default, it will boot nearly everywhere, just like a live CD/USB does. Just hook it up and it will work. However, if you install custom kernel drivers (like proprietary graphics drivers), you might get trouble if the computer you're booting from doesn't have that hardware.
USB
option. – marinus May 18 '15 at 05:52