Linux supports a wide variety of backup media.
The canonical backup media is magnetic tape. There are a wide variety of tape drives available, and Linux supports the majority of them.
Magnetic tape usually has a high storage capacity, but is slow to access relative to the speed of hard drives, and the data stored on it can only be accessed sequentially. This means that if you have something backed up onto tape, you can only restore it by going through the whole tape, from the beginning, until you reach the end of what you want to restore.
This means that tapes are generally only good for archival purposes, or for restoring entire system, rather than restoring individual files.
Another popular backup media are write-able CD-ROMs. This allows you to backup a small amount of data, but you have random read access to it, which means that it's quicker and easier to restore from.
With the advent of write-able DVDs, this allows you to store even larger amounts of data on a single disc.
As the price of hard drives has dropped dramatically, it is becoming popular to simply keep your backups on a large, but perhaps slow and relatively cheap, hard drive, usually in a separate machine. Again, this is handy because you will have random read access to your data, and won't have to fiddle with inserting and removing magnetic tapes or CDs.
Which option you decide to go with will depend on a number of factors:
the amount of data you want
the desired time to perform a restore
No matter which media you decide to go with, it is important to remember that the media holding your backed up data should be stored as far away as possible from the system that you wish to keep backed up!
Imagine that you'd been regularly backing up your server, only to lose all your data as well as the backups because you had the magnetic tapes sitting on top of the server chassis when lightning struck your computer room!
For this reason, companies with critical data often make two backups. One is kept on site, for quick and easy restores, while another copy is taken off-site, to guard against site-wide disasters.
Backup media that is kept on-site should be kept in a fireproof safe, to guard against fire and water damage.