Technology will increase, but people will always stay the same...

QDB: Quote #244321

Scary... and the really frightening part is that you KNOW there are people out there like that...

hdparm is the stuff!

So I've been doing some forensic work lately and have been tinkering with dd. I found the Operator distro (which rocks) and has the AIR (Automated Image & Restore) frontend for dd. As I was testing various scenarios imaging my laptop over to a windows system via nc, I noticed that with AIR I was moving data at 80Mbit or so. Just doing the straight dd command line it was only running at 22Mbit.

Here is the command line I was using:

dd if=/dev/hda conv=noerror,notrunc,sync | nc xxx.xxx.xxx.xxx xxxx

After much tinkering with the command line, thinking it was possibly some of the "conv" options, it turns out that when AIR starts, it does an "hdparm -d1 /dev/xxx" on all devices it finds. This turns on DMA access which does an amazing job at increasing HD performance. Here is a snippet of the performance changes:

root@Operator:/ramdisk/home/knoppix# hdparm -tT /dev/hda

/dev/hda:
Timing cached reads: 1832 MB in 2.00 seconds = 916.00 MB/sec
Timing buffered disk reads: 8 MB in 3.15 seconds = 2.54 MB/sec
root@Operator:/ramdisk/home/knoppix# hdparm -tT /dev/hda

/dev/hda:
Timing cached reads: 1836 MB in 2.00 seconds = 918.00 MB/sec
Timing buffered disk reads: 10 MB in 3.43 seconds = 2.92 MB/sec
root@Operator:/ramdisk/home/knoppix# hdparm -d1 /dev/hda

/dev/hda:
setting using_dma to 1 (on)
using_dma = 1 (on)
root@Operator:/ramdisk/home/knoppix# hdparm -tT /dev/hda

/dev/hda:
Timing cached reads: 1832 MB in 2.00 seconds = 916.00 MB/sec
Timing buffered disk reads: 110 MB in 3.01 seconds = 36.54 MB/sec
root@Operator:/ramdisk/home/knoppix# hdparm -tT /dev/hda

/dev/hda:
Timing cached reads: 1808 MB in 2.00 seconds = 904.00 MB/sec
Timing buffered disk reads: 100 MB in 3.05 seconds = 32.79 MB/sec


As you can see, quite a difference in the buffered disk reads. This allowed me to run dd from the command line at wirespeed!

It should be noted of course that this is only relevant to IDE systems.

Here is a great link on it... Speeding up Linux Using hdparm

Now I have to make sure that DMA is enabled on all my Linux boxes...

Deep Linux Thoughts...

So... I've been experimenting with a myriad of different OS's lately... Everything from various BSD's to a whole assortment of Linux flavors.

I started my journey away from "Winblowz" a long while back with Redhat 6.0. After about 2 hours and multiple failed installation attempts, it occurred to me that Linux was not for the uninitiated. I had wanted to just get it up and running so I could "tinker" with it and learn more. Unfortunately, the act of getting it up and running was WAY above my level. So back to Windows I went.

Later I tried Redhat again at 7.x and 8.x and each time gave up frustrated.

Then came Redhat 9.0. This was the first Linux OS that actually worked for me out of the box. I got it setup and tinkered with it quite extensively. As I learned more, it occurred to me that generally, things were just SO much more complicated in Linux. The simple act of installing a program which in Windows consisted of hitting "Next" a bunch of times, turned into "./configures" and "make's" and "make install's" Oh my! And heaven help you if it didn't compile right... or worse yet, it does compile, but somehow still isn't recognized....

And then of course came "dependencies". Now we had to compile twenty different smaller packages paying close attention to specific versions just to get a particular program running... I hated to admit it, but Windows was looking better every hour I spent staring at a broken makefile wishing I had hair to pull out.

Then came Kyley.

And he brought with him Mandrake. And it was good.

But......

What's this? It's not free? Oh, it's kinda free? There's a "subscription"??? To a club?? But, but... it's Linux... it's supposed to be free?!?!? Oh, if you look really hard, you can download a "free version." But what's the difference between the free version and the "less" free version? Ah... more RPM's... hmmmm.....

Well life continued with Mandrake 9.2... then 10.0... then 10.1.. and finally 10.2. After installing 10.2 with the "free" version it took 2 days to get Nessus working. Dependencies from hell... RPM's that require RPM's that are two new to be up on RPMfind.net. And how will we get these mystery RPM's that are new enough to work with the required RPM's for Nessus... Ah... it all became clear... the necessary RPM's were on the "Mandrake Club" distro. And you have to shell out $$$ for that.

Now I don't know about everybody else, but it seems WRONG to have to PAY for Linux. So I managed to "locate" a copy of "Mandrake Club 10.2 DVD" and got Nessus working.

But something had changed forever... I was using a commercial Linux.

Somehow, at least to me... having to PAY for Linux was just wrong. There had to be a better solution....

So entered Marc.

I queried Marc about the various flavors of Linux... and even BSD. I wanted a FREE OS. Truly free... and not some horrible one with some crappy packaging system and such... An OS with good packaging... and a good package update system... and I wanted it all for FREE. The way it was supposed to be.

So entered a little red guy with horns...

Marc explained that BSD's had a very good packaging system. And better yet, they were FREE. Truly free.

So I debated for a bit whether to install OpenBSD or FreeBSD. In the end it seemed that FreeBSD had more support and packages etc. So I grabbed a copy and installed it via VMWare.

Whoa...

Boy was it ugly.

Gone were my pretty installation GUI's of Redhat and Mandrake. Now we were in text mode and crappy ASCII menu's. Oh well, that's the price I guess. So I continued on... and after a bit of time got it up and functioning... but wasn't completely sold. It was difficult to get around... VERY rough around the edges... It seemed that overall it was pretty decent, but just wasn't my style. Too much was different. I had just been getting my head around Linux and where everything was and how it all worked. Now it was like starting from scratch.

Unfortunately, I just didn't have the time to re-learn ANOTHER OS right at that moment.

So I went back to Marc.

Seeking a flavor of "Linux" that was truly FREE and fully functional.

He described to me a wonderful geek couple who had created their own version of Linux. Their names were Deb and Ian.

So off I went to Debian.org to learn more. It sounded too good to be true. A TOTALLY free OS. No subscriptions, no differing free/non-free versions, just one big fully supported OS.

I went to download Debian and was balked when I hit the ftp download area. There were 14 CD's! Well after reading a bit, it was said that a person could just get the first CD and then Debian would just pull whatever it needed after that. Could it be? What a novel concept. Only grab what you need. I suspiciously started the installation with only ONE CD.

Debian wasn't nearly as polished as the other OS's, the menu's were still ASCII based etc... but the installation process was totally different. Instead of the polished installs of Redhat and Mandrake, which were basically like installing Windows, Debian was.... Interactive.

One of the major issues I had with Redhat and Mandrake were that, like Windows, they installed a mountain of crap that you didn't need. Now of course a seasoned Linux user... or a Newbie Linux user with a lot of time on their hands, can go through and sift out the unnecessary services. But I just didn't like the concept of an OS adding a bunch of stuff just on the off chance you might need it. And I didn't want to kill inordinate amounts of time trying to research every little process that was running...

Debian was the answer to my prayers. The install process took a LONG time... It asked me a million questions... many of which I had NO idea on. However, with every question, it would have an option that said, "If you're not sure, choose X". When my installation was finished I rebooted and Debian began it's first boot.

Wow...

It was fast... not just a little fast... DAMN fast.

It was booted up faster than any OS I've had... And everything worked... NIC, sound, Display... it all worked great. As I looked through the OS, it occurred to me that very few things were running. It only had what I wanted running. Novel...

So next came the big test. How much of a pain in the ass was it going to be to install Nessus... Nessus itself normally installs just fine. But for me, the installation process for the numerous dependencies tends to be the "benchmark" for a Linux Distro.

I went and did a little research and got more familiar with the package install process. Once I thought I had a good handle on it, I went to a shell and issued the following command:

apt-get install nessus nessusd

Before my eyes, the most amazing thing happened. Apt-get went out and found all the dependencies for me, downloaded them and installed them on the fly!! All of 60 seconds later, the brand new current version of Nessus was fully installed.

I sat and stared in shock. I thought of the countless hours I had spent trying to get Nessus working on various Linux implementations. This had taken 5 !@#$@! minutes!!!

I thought it was too good to be true, but nessus fired up like a champ!

So decided to see what other security tools etc it had. I tried dsniff, I tried arpwatch, driftnet, snort, mysql, acid, firefox...

It was amazing... it all installed without a hitch... even the Snort/MySQL/PHP/Apache install took all of 30 minutes. It was paradise. Hell, it was easier to install stuff on here than on Windows!

As one would imagine. I am now a hardcore dedicated Debian guy. It rocks.

I continued on in Debian bliss for quite a while... I even got bold. I attempted to re-compile my kernel. Success! I had now proudly reached "Intermediate Linux User" status!

The interesting thing I learned during the kernel recompile, was that the kernel that shipped with all my previous versions of Linux/BSD wasGeneric. They weren't tuned at all to my computer. I had assumed that when we went through the install process it had determined what my computer hardware was and optimized itself. This was not the case! The kernel I had been running on was set to a Pentium Pro! This seemed very strange to me, but after some conversations with Marc, it became clear that this was really the only way to make sure you could get Linux up and running on most systems. But...

As always, there was an exception... Gentoo.....

Marc described a mystical dream like Linux... Where all the programs were compiled and tweaked to your computer. Where the kernel was compiled during installation so it would be perfect for you system. A Linux where all programs had been perfectly optimized and tweaked as they installed.

Sounds great!

So on a random Saturday as I was tinkering around at home, I went to the dreamworld.

www.gentoo.org

I enjoyed the gentle pastels of the website. The soft curves of the Gentoo logo lulled me in. It was bliss. I downloaded the little tiny 52MB install CD for Gentoo. It downloaded in such a short time... it was wonderful... this would be the "perfect" Linux. I setup my VMWare machine... pointed it towards the distro CD and fired it up. VMWare booted up smoothly... found the CD and all was well with the world. Gentoo began it's sirens song. Suddenly, there was a beautiful calming backdrop of light purples and pinks. It was wonderful. Things had become ethereal. I sat their gazing at the wonderful colors, but was distracted by this blinking from the bottom of the screen...

#_ _ _ _ _ _

The cursor was blinking down at the bottom. How odd.

Surely a wonderful purple and pink GUI will pop up prompting me for simple friendly questions. The words "I love you... You love me..." Were mysteriously rolling around in my mind...

I waited...

and waited....

And much like the gut wrenching nausea generated by that ill-fated children's dinosaur program, so Gentoo had created in me a gastrointestinal distress.

There was no GUI...

There was no menu system....

There was nothing...

Just that damned blinking cursor and those cursed purple and pink shades of annoyance in the background.

$%!$#@!

Something must have gone wrong...

I took a deep breath and downed the VMWare Machine.

I checked all the settings.... surely there must have been a conflict of some kind that had caused the system to hang before it brought up it's install menu....

Everything checked out... I started it up again... Things looked good... it started its nice little boot... ahhh.... look at the pretty colors.....

&#@!$!

#_ _ _ _ _

There it was again... just a cursor... nothing more...

Ok, maybe I just have to manually start up the install script... not too bad...

I did an "ls"

Just a regular dir structure... hmmmm?

There was nothing there... nothing that said "Install" or "Setup"

Something was very wrong... I went back to the Gentoo.org site... Damn annoying shitty ass graphics... where the hell was the install guides... ah, ok... here we go, Gentoo Handbook. Ok, right, stage one install, right... that's what I want...

OH MY GOD....

As I read through the Gentoo Handbook, I realized the truth behind Gentoo... This was not the beautiful fluffy version of Linux... This was the Linux from HELL!!!!!! This was the Linux that gives unsuspecting Linux NOOB's NIGHTMARES! There was no menu system... There was no wizards... this was down and dirty hardcore Linux.... This was the Linux of the early 90's in a clever purple and pink disguise... Before packages... before anything...

I broke into a cold sweat...

OK... get a hold of yourself... it's got an install doc... I can follow a doc... no biggie...

So I began.... working my way step by painful step through the install...

It was horrible... every... little... detail... manually... configured...

After about an hour, I got to a point in the script where you chroot and start Gentoo going.... and kaboom... it exploded....

I must have done something wrong???

I repeated the process....

twice....

Finally I went out to the web and started scouring Google for answers to my error message... It turns out that one of the commands that the Gentoo Handbook specifically tells you to perform... puts 50,000+ lines of code into one of your config files... so after you chroot you can't run the next step as the config file has 50,000 lines of CRAP in it... minor detail....

And of course they don't have vi or any regular editor available... they only have nano....

I pondered exactly which cirle of the inferno I was currently residing in.

2 hours later, I've managed to manually delete the extra 50,000 lines of CRAP from the config file...

I then proceed with the install process....

and proceed...

and proceed...

2 days later I was still compiling....

Sometime after the 3rd day of compiling, I powered down the VMWare Machine... and said goodbye to the evil that is known as Gentoo....