Secure-ish Workaround for Certbot http validation on Ubuntu

LetsEncrypt is a great tool for securing your non-production resources. I personally use it for securing my Unifi controller in AWS along with my NGINX server for my entertainment services. Certbot only has really two methods of validating the certificate renewal at the time of this writing, DNS and HTTP. DNS can be useful if you wish to update your TXT records frequently (API or manually) but clearly the more convenient is HTTP.

The problem I have with HTTP validation is, well it's HTTP and I do not want to keep an unsecured port open on my Ubuntu server perpetually. How can we keep our server secured in between certificate renewals? This posting will show you the required steps to make modifications to the systemd service for Certbot if you are using firewalld on your server.

If you are using Certbot's built-in systemd methods for renewing your certificate you've got a couple things you are in luck because systemd allows for 'ExecStartPre' and 'ExecStartPost' invocations which we can bookend around the 'ExecStart' of /lib/systemd/system/certbot.service . First we will tell systemd to add a new firewall rule to allow http to the appropriate firewall zone before we run the certbot renew command. After the renew command is finished we'll tell systemd to remove the temporary allowance. Simple right?

1) sudo vim /lib/systemd/system/certbot.service
[Service]
Type=oneshot
ExecStartPre=/usr/bin/firewall-cmd --add-service=http --zone=public ## add this line
ExecStart=/usr/bin/certbot -q renew
ExecStartPost=/usr/bin/firewall-cmd --remove-service=http --zone=public ## add this line

2) reload the daemon
sudo systemctl daemon-reload

3) profit

That's it. Those two lines in the systemd service file will open up http and then close it afterwards. You may test it out by commenting the ExecPostStart line to make sure it adds it in there, reloading the daemon, and then running the certbot.service. After you run the service you should see the public zone still has http listed as an available service
ubuntu@ubuntu-server:~$ sudo firewall-cmd --list-all --zone=public
public
  target: default
  icmp-block-inversion: no
  interfaces:
  sources:
  services: dhcpv6-client http https ssh
  ports: 6789/tcp 32400/tcp 33400/tcp 5050/tcp 8989/tcp
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:
And if you repeat the process by commenting out the ExecPreStart and allowing the ExecPostStart to execute you'll see the service is gone.

I'm looking forward to getting no more notification emails about a certificate expiring in 20 days without leaving my server exposed in the meantime. Cheers!

Adding Lets Encrypt to Unifi controller

Hey y'all.

I've had my EC2 t2.micro around for a long time and have always operated with a self-signed certificate. It hasn't bothered me much but it would be nice to get rid of the warning I always receive when logging in. With the advent of CAs such as LetsEncrypt there's really no reason to keep fumbling on the certificate. I had been casually searching on how to work the cert in and FINALLY found a good guide here, Larry Land's tutorial. I'm going to skip some of the information he wrote (he included how to install and setup the controller to begin with) and re-post what I used to get mine going.

Step 1: Generating the signed certificate with Lets Encrypt

Lets install Lets Encrypt now. Reminder that this needs to be done on this server, not your local machine. We’ll be using certbot and essentially the instructions there.

wget https://dl.eff.org/certbot-auto
chmod +x certbot-auto
./certbot-auto

That last line will configure certbot and also install some dependencies.

Now, using certbot, we generate the signed certificate. So lets run the wizard:

./certbot-auto certonly

Select option 2 (to use a temporary webserver), then enter your email (so you get alerts if things go wrong), agree to the agreement, then finally type in your domain name (along with the subdomain). If everything went well you should get a Congratulations message.

Part 2: Load the certs into the services

The Ubiquiti services are Java-based and they use the Java Keystore as a way of storing the private keys and certificates. We first need to generate a PKCS #12 certificate from the raw ones we just received:

sudo openssl pkcs12 -export -inkey /etc/letsencrypt/live/mysubdomain.mydomain.com/privkey.pem -in /etc/letsencrypt/live/mysubdomain.mydomain.com/fullchain.pem -out /home/ubuntu/cert.p12 -name ubnt -password pass:temppass

Again, don’t forget to replace mysubdomain.mydomain.com with your domain name. Everything else can remain as-is.

Now for each service you’ll need to load the PKCS #12 certificate into its own keystore.

• Unifi:

  sudo keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore /var/lib/unifi/keystore -srckeystore /home/ubuntu/cert.p12 -srcstoretype PKCS12 -srcstorepass temppass -alias ubnt -noprompt

Basically all that’s different is the keystore location of the service, and the password Ubiquiti uses to protect it.

Finally, delete the PKCS #12 files (since they’ve already been imported), and restart the services (as appropriate)

sudo rm /home/ubuntu/cert.p12
sudo /etc/init.d/unifi restart

Certificates must be renewed every (3) months. What else do you think crond is for :) I put the script into /etc/cron.monthly/ as that is often enough.

Part 3: Automating Lets Encrypt certificate renewal

As mentioned before, Lets Encrypt certificates only last 3 months. As such, we’ll need to get this machine to attempt to renew the certificates monthly and then place the new certs back into services. It’s essentially doing parts C and D on a scheduled job using cron. The renewal will fail fast if no renewal is necessary.

Create a new file /etc/cron.monthly/renew_lets_encrypt_cert.sh and customize it according to what you used in Parts C and D. No sudo needed since cron will run it automatically as a super user. Use full paths to files. Here’s an example:

# Get the certificate from LetsEncrypt
/home/ubuntu/certbot-auto renew --quiet --no-self-upgrade

# Convert cert to PKCS #12 format
openssl pkcs12 -export -inkey /etc/letsencrypt/live/mysubdomain.mydomain.com/privkey.pem -in /etc/letsencrypt/live/mysubdomain.mydomain.com/fullchain.pem -out /home/ubuntu/cert.p12 -name ubnt -password pass:temppass

# Load it into the java keystore that UBNT understands
keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore /var/lib/unifi/keystore -srckeystore /home/ubuntu/cert.p12 -srcstoretype PKCS12 -srcstorepass temppass -alias ubnt -noprompt

# Clean up and use new cert
rm /home/ubuntu/cert.p12
/etc/init.d/unifi restart

Make sure the script is executable:

sudo chmod +x /etc/cron.monthly/renew_lets_encrypt_cert.sh

Let's check back next month and ensure everything renewed itself properly.

EdgeRouter recovery from start to finish on a Mac

The Ubiquity EdgeRouter Lite (ER3-Lite) router is a very capable router I have grown to lean on for most of my customer installs. It has a great value, good community support, and has a lot of features - even more if you dive into the CLI -  which business customers can take advantage of. I've covered a few other configuration walk throughs on here so I won't go into any detail on what it can do or why I like it.

The point of this post is to put together from start to finish how to rebuild your ER3-Lite router if the USB storage goes south and you have a Mac computer at your disposal. I will provide links back to the original instructions in the post so you can go find more information if there is a bit I either glaze over or you want to follow up on more of the solution. The reason I'm writing this one myself is to give credit to those who have come before me and to bring multiple posts into one spot for someone else.  Ubiquity community EdgeMax rescue thread: EdgeMax rescue kit (now you can reinstall EdgeOS from scratch)

There are two pieces of extra equipment you're going to need if you follow these instructions plus one compatible USB storage drive: 1 serial console cable (sometimes called a rollover cable) and a USB-Serial adapter. For my use I had a couple console cables left over from Cisco switches and a Keyspan USB-Serial adapter already. I had good luck and good value/performance with Kingston Digital 16GB DataTraveler SE9 USB 2.0 Flash Drive Pack of 2 (KW-U4616Z02-8A). There are others out there which can be used but your success may not be guaranteed. This particular model works well and has been proven many times over. If this post is more than a few months old, the mentioned storage drive is no longer available, and you're looking for guidance on what to buy I suggest you reference back to the original Ubiquity community thread for what others are using. If I run out of stock of the drives and find another I'll update this post.

*** NOTE *** Previously unusable drives like the CruzerFit drives from Sandisk are confirmed to be usable if you follow the last bit of this post where you modify the bootcmd of the router to include a usb reset and a couple sleeps.

First, we need to install the USB-Serial drivers and then get the console to work via terminal. After you've installed the appropriate drivers for your USB-Serial adapter lets drop onto the CLI via Terminal and look at what device you have. The device is going to be listed under /dev/cu.something. Mine happens to have Keyspan in the name. The command we use to execute the connection will be:

screen /dev/cu.KeySerial1 115200

I suggest having the cable already inserted into your powered up EdgeMax and the new storage drive installed before executing the screen command. Once the screen command is showing you a prompt you should see something like this:

Octeon ubnt_e100#

You're ready to start the building process BUT we still need to fire up an FTP server on the Mac and serve up the image for tftpbooting. Download the latest available EMRK from here and place it into /private/tftpboot/.

open /private/tftpboot/ 

sudo chmod -R og+rX /private/tftpboot/
sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist
sudo launchctl start com.apple.tftpd


## To shut down tftp server
sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist

Once you get your file downloaded and tftp server running keep following the instructions on the original thread, EdgeMax rescue kit (now you can reinstall EdgeOS from scratch). I preferred to plug my eth0 into my existing bench network (which allowed it to get out to the internet) and the console cable into the console port. This allowed me to also use dhcp from the bench network and to pull down the newest available EdgeRouter image from the internet. My commands were as follows:

dhcp
set bootfile emrk-0.9c.bin
set serverip <machost IP>
tftpboot;bootoctlinux $loadaddr

Once the file successfully pulled from my Mac's TFTP server and loaded up I was able to find myself looking at the EMRK loading away. I again used DHCP to configure eth0 which again made things easier for me. When at the EMRK> prompt I suggest deleting all existing partitions via the parted command. I wasted a little bit of time not doing this, expecting the EMRK to do it for me. It may work for others but what I'm giving you advise on will work every time.

parted -s /dev/sda rm 1
parted -s /dev/sda rm 2

Now that it's completely clean we can let the emrk-reinstall command do its magic. Before we continue lets get the URL from Ubquity to the newest available firmware release, Ubiquity Networks Downloads ER3-Lite. Save the URL in your clipboard as we will need it shortly. Upon execution of the emrk-reinstall script it will shortly ask for the URL to the firmware. The underlying execution uses a curl command to pull it in. Since the URL from Ubiquity is 'https' we must add '-k' to the beginning of the supplied URL so it will ignore the certificate error. I'm including an example below:

kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with writeback data mode.
Enter EdgeOS image url: -k https://dl.ubnt.com/firmwares/edgemax/v1.10.x/ER-e100.v1.10.1.5067571.tar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 68.3M  100 68.3M    0     0  8319k      0  0:00:08  0:00:08 --:--:-- 6318k

This should finally result in a finished image. If it does not please consult the original links for further troubleshooting. If you wish there are also tutorials on how you can expand the install script from the original 2GB partition to use the full size of your drive. I did this but it's outside the scope of this post.

You're ALMOST ready to say you're finished with this one but there's one more gotcha I'm going to post here to make this a 100% post. The USB storage available to us seems to be of different type than the ones used by the OEM. This can result in the router not correctly initilizing the storage upon boot reliably. You may find yourself sitting at a prompt as if all of your work was for nothing the first time the power is re-applied to the router. (It happened to me and is why I'm writing all of this for the community.) The last bit you need is here, List-of-Compatible-USB-drives post 24. Edit the bootcmd to allow the usb storage device to be reset before being accessed by the boot process. If this is done it'll fix the problem with dissimilar storage devices from a non-OEM. First set environment variable bootcmd with this command:

Octeon ubnt_e100# setenv bootcmd sleep 1\;usb reset\;sleep 1\;$(bootcmd)
Note all this must be on single line !
Next we must be sure that changes are accepted with this command:

Octeon ubnt_e100# printenv
Octeon ubnt_e100# saveenv

The router should now boot reliably. That's it! There are a few other shortcuts to all of this process made possible through other script writers but they are all doing the same thing as I've recounted here. The problem with always working through someone else's shortcuts is you never really know how to fix something on your own. Certain exceptions here are the creation of a tftpboot image and the tool itself we used, EMRK. I didn't think this should be something I should learn how to do on my own as it was going to be well outside the scope of anything I would be looking to accomplish. I hope this helps someone else. Not all of these links and procedures were in one place that I found over the course of a few days but now they are. Enjoy!

Update on using squidGuard Webproxy for Ubiquity

For those who have read my previous blog post about getting my feet wet with the webproxy package installed on our EdgeOS based routers I have some updated notes to help out with a few things I didn't find well documented and readily available on the internet.

1) how to un-disable the webproxy url-filtering.
   If you needed to disable the webproxy for one reason or another it's not that difficult. The command to do so is built into the auto-complete command set:
  configure
 set service webproxy url-filtering squidguard disable
 commit

What isn't documented very well in my searching was how to UN-disable it after you've made your changes and are ready to get the webproxy back online. It took me some thinking on my own, as I'll often do when stuck on a problem, but my logic was sound and I had nothing to lose.
  configure
 delete service webproxy url-filtering squidguard disable
 commit

Success! The service was back online and I didn't have to rebuild my configuration.

2) how to save your /var/log/ mount point after turning on DPI.
After enabling DPI, outlined in my other post, a lot of logging is turned on. Logging is good if you're wanting to do some checking on your traffic but this gets excessive when you have a busy router. Your /var/log/messages logs will overrun the mount point quickly of some housekeeping isn't done to curb this. Via the GUI you'll want to go to your rulesets and unchecked logging for all of your accept rules. If you are special and you have a server to accept all of your logs for you need you not worry.

3) Making a separate proxy profile for separate subnets
Guest Wi-Fi subnets rarely need the same allowances as an internal network may so if a guest complains about not being able to get to break.com is not a big deal. However, maybe someone internal needs it. What do you do?

You could make exceptions for the entire router for individual sites but over time the list could grow and then you'd be opening it up for everyone. Lucky for you source-groups are able to handle the task. With a source group you can leave the default allows and blocks on for all but make changes by exception source-groups.

set service webproxy url-filtering squidguard source-group guests address 192.168.2.2-192.168.2.254

set service webproxy url-filtering squidguard rule 10 source-group guests

set service webproxy url-filtering squidguard rule 10 allow-category adult

Once this has been committed and saved you now have allowed the adult category, which is what category break.com belongs to, sites for those IPs in the internal network. This same method can be utilized for other exceptions and restrictions.

If you have questions or comments please leave them below.

Ubiquity EdgeOS Deep Packet Inspection engine (DPI)

Recently I wrote a blog about how to use the built in package squidGuard to block content passing through EdgeOS routers. There is a more efficient way to block some categories though, DPI.

DPI is hardware offloaded and does not drag your CPU down as much on a busy router. The number of categories you can block are limited though:

Business                                  TopSites-Games
Bypass-Proxies-and-Tunnels  TopSites-Health
File-Transfer                           TopSites-Home
Games                                     TopSites-KidsnTeens
Instant-messaging                   TopSites-News
Mail-and-Collaboration          TopSites-Recreation
P2P                                         TopSites-Reference
Remote-Access-Terminals     TopSites-Regional
Security-Update                     TopSites-Science
Social-Network                     TopSites-Shopping
Stock-Market                        TopSites-Society
Streaming-Media                  TopSites-Sports
TopSites-Adult                      Voice-over-IP
TopSites-Arts                        Web
TopSites-Business                 Web-IM
TopSites-Computers

Most notable of the categories I like to use are the Bypass-Proxies-and-Tunnels and P2P. With making a separate firewall rule set for these two you can keep casual users from hopping on your wifi and downloading illegally data you would be left holding the bag for. By Turing on the DPI feature you can also see what protocols and categories are most used on your network by your top talkers.

With the below lines of configuration I can say it successfully has caught and stopped users from downloading torrents on a particular network I manage.

set system traffic-analysis dpi enable
set system traffic-analysis export enable
set firewall name DROP_SITES default-action accept
set firewall name DROP_SITES description "Drop Junk"
set firewall name DROP_SITES enable-default-log
set firewall name DROP_SITES rule 10 action drop
set firewall name DROP_SITES rule 10 application category P2P
set firewall name DROP_SITES rule 10 log enable
set firewall name DROP_SITES rule 10 description ‘P2P’
set firewall name DROP_SITES rule 15 action drop
set firewall name DROP_SITES rule 15 application category Bypass-Proxies-and-Tunnels
set firewall name DROP_SITES rule 15 log enable
set firewall name DROP_SITES rule 15 description Bypass-Proxies-and-Tunnels
set interfaces ethernet eth0 firewall in name DROP_SITES

After setting it all up you'll be able to see the rules stats counting up as traffic is passed. Logged drops are put into the default /var/log/messages log. As it is put into the general log I strongly suggest you turn compression on for  logrotated. Otherwise/var/log will fill up quickly lest you send all of your logs to a remote server.

WSUS Server crapped the bed after installing KB3159706

I have several domain controllers I manage for local businesses and most of them I run WSUS on the hyper-v host. Lately I've had a run in with KB3159706 which was not pleasant. After it's installed I get the following error in Windows Event Viewer: 
"Login failed for user 'NT AUTHORITY\NETWORK SERVICE'. Reason: Failed to open the explicitly specified database 'SUSDB'. [CLIENT: <named pipe>]"

If you look on the internet for a bit you'll find several solutions saying to remove the referenced KB. I say nay to that.

Basically the steps boil down to:

1. Open an elevated Command Prompt and run "C:\Program Files\Update Services\Tools\wsusutil.exe" postinstall /servicing
2. Enable HTTP Activation under .NET Framework 4.5 Features in the Server Manager Add Roles and Features Wizard
3. Restart the WSUS service

No need to uninstall KB3159706.

Source: https://social.technet.microsoft.com/Forums/windows/en-US/233a6b37-5919-4a9e-9538-aa0e7bd7111a/wsus-console-service-cant-start-reason-failed-to-open-the-explicitly-specified-database?forum=winserverwsus

Content filtering along with Ad-blocking using Unifi EdgeOS

This summary is not available. Please click here to view the post.

Dumb questions I really enjoy as funny jokes

A stitch in time saves nine what? After eating, do amphibians have to wait one hour before getting out of the water? After they make Styrofoam, what do they ship it in? --Steven Wright Are female moths called myths? Are part-time band leaders semi-conductors? Are there a lot of virgins in the Virgin Islands? Are there any unguided missiles? Are you breaking the law if you drive past those road signs that say "Do Not Pass"? Are you telling the truth if you lie in bed? Before they invented drawing boards, what did they go back to? Can a stupid person be a smart-ass? Can fat people go skinny-dipping? Can you buy an entire chess set in a pawnshop? Can you get cavities in your dentures if you use too much artificial sweetener? Could crop-circles be the work of a cereal killer? Crime doesn't pay... does that mean my job is a crime? Day light savings time - why are they saving it and where do they keep it? Did Noah keep his bees in archives? Do blind dogs have seeing-eye humans? Do blind Eskimos have seeing-eye sled dogs? Do boxer shorts box? Do cemetery workers prefer the graveyard shift? Do clowns wear really big socks? Do crematoriums give discounts to burn victims? Do files get embarrassed when they get unzipped? Do fish get thirsty? Do hummingbirds hum because they don't know the words? Do hungry crows have ravenous appetites? Do infants enjoy infancy as much as adults enjoy adultery? Do jellyfish get gas from eating jellybeans? Do mass murderers kill only in church? Do people in Australia call the rest of the world 'up over'? Do pilots take crash-courses? Do Roman paramedics refer to IV's as "4's"? Do Scottish Terriers get Scotch Tape worms? Do stars clean themselves with meteor showers? Do steam rollers really roll steam? Do television evangelists do more than lay people? --Stanley Ralph Moss Do vampires get AIDS? Do vegetarians eat animal crackers? Do witches run spell checkers? Do you need a silencer if you are going to shoot a mime? Do you realize how many holes there could be if people would just take the time to take the dirt out of them? Do you think that when they asked George Washington for ID that he just whipped out a quarter? --Steven Wright Does a man-eating shark eat women, too? Does an analyst have to be anal? --Adam Rifkin Does killing time damage eternity? Does that screwdriver belong to Phillip? Does the Little Mermaid wear an algebra? Ever notice how irons have a setting for "permanent" press? I don't get it. --Steven Wright Have you ever imagined a world with no hypothetical situations? Have you ever seen a toad on a toadstool? Have you ever talked into an acoustic modem? Have you ever wondered? How can someone "draw a blank"? How can there be self-help "groups"? How can you tell when it is time to tune your bagpipes? How come chocolate milk doesn't come from brown cows? How come I can pick my ears but not my nose? How come wrong numbers are never busy? How dead is the Dead Sea? How did a fool and his money get together? How did the man who invented cottage cheese know he was done? How do I set my laser printer on stun? How do they get a deer to cross at that yellow road sign? How do they get the "Keep off the Grass" sign on the grass? How do you get off a nonstop flight? How do you know if honesty is the best policy unless you've tried some of the others? How do you know when you've run out of invisible ink? How do you throw away a garbage can? How do you write zero in Roman numerals? How does a person with a lisp pronounce that word? How does a thermos know whether a drink should be hot or cold? How does it work out that these people always die in alphabetical order? How does the guy who drives the snowplow get to work in the mornings? How is it possible to have a "civil" war? How is it possible to run out of space? How long is the long arm of the law? How many people does it take to change a searchlight bulb? How many weeks are there in a light year? How much can I get away with and still go to heaven? How much milk is there in the Milky Way? How much wood could a woodchuck chuck if a woodchuck could chuck wood? How old would you be if you didn't know how old you was? --Satchel Paige If 7-11 is open 24 hours a day, 365 days a year, why are there locks on the doors? If a candle factory burns down, does everyone just stand around and sing "Happy Birthday?" If a fly has no wings would you call him a walk? If a food processor slices and dices food, what does a word processor do? If a hen and a half can lay an egg and a half in a day and a half, how long would it take a monkey with a wooden leg to kick the seeds out of a dill pickle? --Tom Robbins If a jogger runs at the speed of sound, can he still hear his Walkman? If a mime commits suicide, does he use a silencer? --Steven Wright If a mirror reverses right and left, why doesn't it reverse up and down? If a picture is worth a thousand words, what is a picture of a thousand words worth? If a pig is sold to the pawn shop, is it a ham-hock? If a pizza place sells pizza by the slice, is there a guy in the back tossing a triangle in the air? --Steven Wright If a pronoun is a word used in place of a noun, is a proverb a word used in place of a verb? If a tree falls in the forest, does the earth scream out in pain? If a turtle doesn't have a shell, is he homeless or naked? If a vegetarian eats vegetables, what does a humanitarian eat? If a word in the dictionary were misspelled, how would we know? If absolute power corrupts absolutely, does absolute powerlessness make you pure? --Harry Shearer If all the nations in the world are in debt, where did all the money go? If all the world is a stage, where are the audience sitting? If an orange is orange, why isn't a lime called a green, and a lemon called a yellow? If athletes get athlete's foot, do astronauts get mistletoe? If Barbie's so popular, why do you have to buy all her friends? If bees live in an apiary, do apes live in a beeiary? If blind people wear dark glasses, why don't deaf people wear earmuffs? --Steven Wright If cats and dog didn't have fur would we still pet them? If corn can't hear, why does it have an ear? If corn oil is made from corn, what is baby oil made from? If crime fighters fight crime and fire fighters fight fire, what do freedom fighters fight? If everything is part of a whole, what is the whole part of? --Ashleigh Brilliant If flowers don't talk back to you, are they mums? If Fred Flintstone knew that the large order of ribs would tip his car over, why did he order them at the end of every show? --Steven Wright If God can do anything, can he make a rock so big he can't lift it? --George Carlin If God dropped acid, would he see people? If humans get a charley horse, what do horses get? If humans have nightmares, what do horses have? If I melt dry ice, can I take a bath without getting wet? --Steven Wright If I save time, when do I get it back? If Jimmy cracks corn and no one cares, why is there a song about him? If love is blind, why is lingerie so popular? If man evolved from apes why do we still have apes? --Dennis Miller If nothing ever sticks to TEFLON, how do they make TEFLON stick to the pan? If one synchronized swimmer drowns, do the rest have to drown too? If our knees were on the backs of our legs, what would chairs look like? If peanut butter cookies are made from peanut butter, then what are Girl Scout cookies made out of? If rabbits' feet are so lucky, then what happened to the rabbit? If someone with multiple personalities threatens to kill himself, is it considered a hostage situation? If space is a vacuum, who changes the bags? If Superman is so smart why does he wear his underpants over his trousers? If swimming is good for your shape, then why do the whales look like the way they do? If the #2 pencil is the most popular, why is it still #2? If the funeral procession is at night, do folks drive with their lights off? If the product says "Do not use if seal is broken", how are you supposed to open it and use it? If there is no God, who pops up the next Kleenex? --Art Hoppe If time heals all wounds, how come bellybuttons don't fill in? If tin whistles are made out of tin, what do they make fog horns out of? If white wine goes with fish, do white grapes go with sushi? If work is so terrific, how come they have to pay you to do it? If you ate pasta and antipasta, would you still be hungry? If you can wave a fan, and you can wave a club, can you wave a fan club? If you can't drink and drive, why do bars have parking lots? If you choke a Smurf, what color does it turn? If you didn't get caught, did you really do it? If you get into a taxi cab, and ask the driver to drive backwards to your destination, will the cab driver owe you money? If you had a million Shakespeares, could they write like a monkey? If you jog backwards, will you gain weight? If you saw a heat wave, would you wave back? If you take an Oriental person and spin him around several times, does he become disoriented? If you tell a joke in the forest, but nobody laughs, was it a joke? If you throw a cat out a car window, does it become kitty litter? If you tied buttered toast to the back of a cat and dropped it from a height, what would happen? If you try to fail, and succeed, which have you done? If you're born again, do you have two bellybuttons? If your car says Dodge on the front of it, do you really need a horn? If you're a kleptomaniac, is there something you can take for it? If you're in a vehicle going the speed of light, what happens when you turn on the headlights? In court, why do they ask if you swear to tell the truth? If you're planning on lying, do they really think you'll tell them so? Instead of wasting time hunting and cooking, why don't hunters just use flame-throwers? Is "tired old cliché" one? Is a castrated pig disgruntled? Is a halfback more valuable than a quarterback? Is a sleeping bull a bull-dozer? Is a small pig called a hamlet? Is an oxymoron a really dumb bovine? Is drilling for oil boring? Is duck tape made out of ducks? Is it OK to use the AM radio after noon? Is it true that cannibals don't eat clowns because they taste funny? Is that a flying saucer or a pie in the sky? Is the nose the scenter of the face? Is this bullshit or fertilizer? Now that Microsoft is so big, should it be called Macrosoft?

Sexual harassment at work-is it a problem for the self-employed? --Victoria Wood Since Americans throw rice at weddings, do Asians throw hamburgers? Since there is a speed of light and a speed of sound, is there a speed of smell? The Scarecrow got a brain, Tin Man got a heart, Lion got courage, Dorothy got home, what did Toto get? Was the pole vault accidentally discovered by a clumsy javelin thrower? What are imitation rhinestones? What do batteries run on? What do chickens think we taste like? What do penguins wear for play clothes? What do people in China call their good plates? What do sheep count when they can't get to sleep? What do they call a French kiss in France? What do they call coffee breaks at the Lipton Tea Company? What do you call a bedroom with no bed in it? What do you call a male ladybug? What do you call male ballerinas? What do you say if you're talking to God, and he sneezes? What does Geronimo say when he jumps out of a plane? What hair color do they put on the driver's license of a bald man? What happened to the first 6 UP's? What happens if you get scared half to death, ...twice? --Steven Wright What happens if you go on a survival course - and you don't pass? What happens if you take No-Doze and wash it down with Nyquil? What happens to an 18 hour bra after 18 hours? What happens when none of your bees wax? What happens when you swallow your pride? What if hell really did freeze over? What would we be using instead? What if someone died in the living room? What if the hokey pokey really is what it's all about? What if you're in hell, and you're mad at someone, where do you tell them to go? What is "Soft Liquor"? What is a "free" gift? Aren't all gifts free? What is a refried bean? Why do they have to fry it twice? What is shaved ice? Did it have hair on it before it was shaved? What is the diameter of a square? What is the speed of dark? What kind of fruit is in Juicy Fruit gum? What makes cheese so confidential that we actually need cheese shredders? What was the best thing before sliced bread? Whatever happened to preparations A through G? What's the sound a name makes when it's dropped? What's the synonym for thesaurus? When cheese gets its picture taken, what does it say? When cows laugh, does milk come out of their noses? When day breaks who fixes it? When dog food is new and improved tasting, who tests it? When I erase a word with a pencil, where does it go? When night falls who picks it up? When people lose weight, where does it go? When something fades in the sunlight, where did the colors go? When they first invented the clock, how did they know what time it was to set it to? When we say our mind wanders - where does it go? When you put a sheet over your head for Halloween, are you a ghost or a mattress? Where are the germs that cause 'good' breath? Where are we going? And what's with this hand basket? Where did Webster look up the definitions when he wrote the dictionary? Where do they get Spring water in the other 3 seasons? Where does the fire go when the fire goes out? Where does the white go when the snow melts? Where does your lap go when you stand up? Where in the nursery rhyme does it say Humpty Dumpty is an egg? Where is Old Zealand? Which is the other side of the street? Who killed the Dead Sea? Who opened that first 'oyster' and said "My, my, my. Now doesn't 'this' look yummy!" Who was the first person to look at a cow and say, "I think I'll squeeze these dangly things here and drink whatever comes out?" Who was the first person to see an egg come from a chicken's butt and think, "I'll bet that would be good to eat? Whose cruel idea was it for the word 'lisp' to have an 's' in it? Why are all blackboards called that when some of them are green? Why are America's parks administered by the Department of the Interior? Why are builders afraid to have a 13th floor, but book publishers aren't afraid to have a Chapter 11? Why are cigarettes sold in gas stations when smoking is prohibited there? Why are hemorrhoids called "hemorrhoids" instead of "asteroids"? Why are highways build so close to the ground? Why are raisins called raisins if they are only dried grapes? Why not just call them dried grapes? Why are some gay people so unhappy? Why are there flotation devices under plane seats instead of parachutes? Why are there interstate highways in Hawaii? Why are they called "stands" when they're made for sitting? Why are violets blue and not violet? Why are you expected to slow down in a speed zone? Why aren't there ever any guilty bystanders? Why buy shampoo when real poo is still free? Why can't you make another word using all the letters in "anagram"? Why can't we tickle ourselves? Why did kamikaze pilots wear helmets? Why did the pot call the kettle black? Why didn't Noah swat those two mosquitoes? Why do ballet dancers dance on their toes? Why doesn't the company just hire taller dancers? --Fred Allen Why do fat chance and slim chance mean the same thing? Why do flamingos stand on only one leg? Why do flammable and inflammable mean the same thing? Why do hot dogs come ten to a package and hot dog buns only eight? Why do mattresses have springs, if they aren't made for jumping on? Why do people always remember where they were when someone famous was killed? Do they feel perhaps they'll need an alibi? Why do people go to the unemployment office to find a job? Why do people park in driveways and drive on parkways? --Larry Anderson Why do people tell you when they are speechless? Why do pigs have curly tails? Why do the signs that say "Slow Children" have a picture of a running child? Why do they call it 'chili' if it's hot? Why do they call it a TV set when you only get one? Why do they call it 'getting your dog fixed' if afterwards it doesn't work anymore? Why do they call it life insurance? Why do they call it quicksand when it sucks you down slowly? Why do they call the piece of wood a two-by-four if it's only 1 3/4" x 3 1/2"? Why do they call Wednesday hump day, when most people get laid on the weekends? Why do they give you a tape with a VCR to tell you how to use it? Why do they make cars go so fast its illegal? Why do they make scented toilet paper? Why do they put Braille dots on the keypad of the drive-up ATM? Why do they report power outages on TV? Why do they sterilize the needle for lethal injections? Why do toasters always have a setting that burns the toast to a horrible crisp, which no decent human being would eat? Why do tourists go to the top of tall buildings and then put money in telescopes so they can see things on the ground in close-up? Why do we call something sent by car a shipment and something sent by ship a cargo? Why do we call them restrooms when no one goes there to rest? Why do we drive on parkways and park on driveways? Why do we have hot water heaters when hot water doesn't need to be heated? Why do we kill people for killing people to show that killing is wrong? Why do we need training bras? What can we teach them? Why do we put shirts in a suitcase, and put suits in a garment bag? Why do we say "a pair of pants" when there is only one article of clothing involved? Why do we say something is out of whack? What is a whack? Why do we sing "Take me out to the ball game," when we are already there? Why do we wash bath towels? Aren't we clean when we use them? Why do wise guy and wise man mean entirely different things? Why do you feet smell and your nose runs? Why do you need a driver's license to buy liquor when you can't drink and drive? Why do you need an appointment to see a psychic? Why does a dishtowel get wet when it dries? Why does a grapefruit look nothing like a grape? Why does Donald Duck wear a towel when he comes out of the shower, when he doesn't usually wear any pants? Why does Goofy stand erect while Pluto remains on all fours? They're both dogs! Why does it take 15 minutes to cook minute rice? Why does the sun lighten our hair, but darken our skin? Why doesn't glue stick to the inside of the bottle? Why doesn't Tarzan have a beard? Why doesn't the fattest man in the world become a hockey goalie? Why don't they call mustaches "mouthbrows?" Why don't they just make food stamps edible? Why don't you ever see the headline "Psychic Wins Lottery"? Why get even, when you can get odd? Why is "abbreviated" such a long word? Why is a boxing ring square? Why is a carrot more orange than an orange? --Amboy Dukes Why is a person who plays the piano called a pianist, but a person who drives a race car not called a racist? Why is a women's prison called a penal colony? Why is brassiere singular and panties plural? Why is clear considered a color? Why is it called a "building" when it is already built? Why is it called lipstick if you can still move your lips? Why is it called tourist season if we can't shoot at them? Why is it considered necessary to nail down the lid of a coffin? Why is it that bullets ricochet off of Superman's chest, but he ducks when the gun is thrown at him? Why is it that doctors call what they do "practice"? Why is it that famous people are always born on holidays? Why is it that night falls but day breaks? Why is it that only adults have difficulty with childproof bottles? Why is it that rain drops but snow falls? Why is it that to stop Windows 95, you have to click on "Start"? Why is it that when you're driving and looking for an address, you turn down the volume on the radio? Why is it you must wait until night to call it a day? Why is lemon juice made with artificial flavor, and dishwashing liquid made with real lemonsWhy is the alphabet in that order? Is it because of that song? Why is the man who invests all your money called a broker? Why is the third hand on the watch called a second hand? --Steven Wright Why is the time of day with the slowest traffic called rush hour? Why is the word dictionary in the dictionary? Why is your index finger the same size as your nostrils? Why isn't phonetic spelled the way it sounds? Why isn't there a special name for the tops of your feet? --Lily Tomlin Why isn't there mouse-flavored cat food? Why was Evelyn Wood in such a hurry? You can't have everything. Where would you put it? You know that little indestructible black box that is used on planes? Why can't they make the whole plane out of the same substance? Copied from here in case they every take the site down. Priceless!