Power Saving GOD Mode on Linux (Part – 2)

This is the advanced guide to Linux power saving. Advanced in the sense that I’ll walk you through the process of learning about the different modules used by your kernel; also how to apply the different power management settings supported by your hardware.

Part of the hacker mindset is to learn, explore & think outside the box which can sometimes lead you to a whole new world of opportunities & at other times can cause you trouble if you’re not careful enough 😉

Alright, so first of all you need to know the devices which are present in your system (e.g graphics card, wireless adapters, etc.) and the modules they use that support power management options.

One way find that out is to use the command: lspci -v

gpuwifi

The command shows you a lot of details but as you can see above, I’ve an integrated intel card running the kernel module i915 & the Wireless Atheros card running ath9k module.

Okay so what you need to do next is learn more about the options supported by these modules.

You can do so by using the following command: modinfo -p [module name]

In my case these are the parameters supported by the module (ath9k) used by my wireless card.

wireless

As you can see, there’s the flag ps_enable which can be activated to enable power management options. By default it’s not activated, and even when I try to enable it through powertop it doesn’t seem to work. So in order to make sure that feature works I’ve to enable it by explicitly stating it in my boot settings.

We’ll get to that part (enabling @boot settings) a bit later but first let’s see the power management options supported by my intel card.

i915

The module i915 has a lot of features but I decided to just focus on the power saving aspect of it.

As you can see there’s 4 main features (i915_enable_rc6, i915_enable_fbc, lvds_downclock, enable_pc8) that you can enable in your boot settings to make them work properly. By default some of these features are probably disabled because it is known to cause problems with certain types of hardware. Therefore you should be cautious about which feature you plan on enabling & double check to see if everything works out fine.

As you can also see in the description of these parameters, most require 1 for enabling, 0 to disable, etc. so use them accordingly.

Let’s get started on setting up these parameters on our boot settings. Personally I use grub as my bootloader which is the default bootloader on a lot of other popular distros like Ubuntu, Mint, etc.

The default location for grub cofig file on my distro is in /etc/default/grub

Open that config file in your favourite text editor make sure you have root priviledge.

Go to the line that says:

GRUB_CMDLINE_LINUX_DEFAULT

There would probably be some parameters set in that line already, but to enable our power management parameters we have to do it like this:

[module name].[module function]=value

e.g:  ath9k.ps_enable=1

Each of the supplied parameters have to be separated by space.Therefore the settings in my grub file looks like this:

GRUB_CMDLINE_LINUX_DEFAULT=”quiet intel_pstate=disable ath9k.ps_enable=1 i915.i915_enable_rc6=1 i915.lvds_downclock=1 i915.i915_enable_fbc=1 i915.semaphores=1″

After you make the changes don’t forget to use the command update-grub otherwise the parameters won’t be set.

Also as you can see from the settings in my grub config file, I’ve set the parameter intel_pstate=disable

From what I’ve heard intel_pstate driver has issues with certain processors or the implementation doesn’t really work that well for the case of power management, etc.

I’ve disabled pstate & I configured tlp to clock the speed of my cores according to the speeds that I want.

These are the configurations on my tlp config file located in /etc/default/tlp

CPU_SCALING_GOVERNOR_ON_AC=ondemand
CPU_SCALING_GOVERNOR_ON_BAT=powersave

CPU_SCALING_MIN_FREQ_ON_AC=800
CPU_SCALING_MAX_FREQ_ON_AC=3200
CPU_SCALING_MIN_FREQ_ON_BAT=800
CPU_SCALING_MAX_FREQ_ON_BAT=2000

CPU_BOOST_ON_AC=1
CPU_BOOST_ON_BAT=0

SCHED_POWERSAVE_ON_AC=0
SCHED_POWERSAVE_ON_BAT=1

I’ve set the minimum frequency of the core down to 800 & I think by default it used to be around 2000. After reboot the changes should take place & you might notice less heating or power usage from your cpu aswell.

Also currently I run 3.14 kernel (did notice slight power improvements), and having made all these changes my power usage dropped to 9W (lowest) & averaging on about 10-11W.

So that’s it for my Linux power saving guide, I hope I’ve been able to share what I’ve learned & hope that it helped some people. If I had this knowledge earlier I’d have been so much happier, would appreciate any feedback.

Thanks 🙂

Advertisements

Power Saving GOD Mode on Linux (Part – 1)

Some people who use Linux on a daily basis have a lot of issues configuring their system to attain the optimal power saving features supported by their hardware (mainly due to lack of experience), which is why the battery life on Linux is not even close to that on Windows. It’s not just about configuring the features, sometimes it can be due to the driver itself lacking good power management support for certain hardware.

Anyway, I’ve been looking around a LOT and finally I think I’ve achieved some of the wisdom to understand how and what to mess around with to get the most out of my laptop’s battery and I wish to share it with you! 🙂

Okay, so there’s a lot of things to look for & there are many tweaks to be made depending on your preferences & hardware. For the basic configuration it should be the same regardless of your hardware, but for advanced configuration I’ll show you the steps towards detecting & enabling the power management configurations in the kernel level manually. There’s a lot of things to write about so I’ve decided to make it into two parts. This part will only cover the basics, on the next part I hope to cover the advanced configuration stuff.

 

Basic Configuration:

First of all, there’s 2 main utilities on Linux that are out there which automatically optimises your hardware settings for power management depending on whether your laptop is plugged to a power source or not.

1) TLP (https://wiki.archlinux.org/index.php/TLP)

2) Laptop-mode-tools (https://wiki.archlinux.org/index.php/Laptop_Mode_Tools)

You have to choose one of the above tools but not both as they conflict with each other. I personally would recommend going with TLP as it has good power management defaults & it automatically does all the things for you where as in laptop mode you would have to configure the settings manually before you can start using it. Which can be quite time consuming but it’s upto you. Please follow the steps in the wiki accordingly, and also note that in case of tlp or laptop-mode you may need to add usb device hardware id’s (for stuff like a mouse, etc.) in the configuration file to make sure they work after your laptop is unplugged from the power source.

 

After installing and configuring one of the above tools, I’d recommend installing Powertop (https://wiki.archlinux.org/index.php/powertop)

Powertop is the main tool that you rely on to get statistics about your laptop’s power usage.

With this utility you get to see which device is using how much Watt of power.

powertop1

 

You can even enable some power management settings for your hardware from within powertop if they’re not enabled already.

powertop2

 

For me personally, since I’m paranoid about security, I blacklisted the bluetooth and webcam modules completely so that they don’t load at boot by any chance as I don’t use those services anyway. That’s also a good way to save up on power. If you’re on systemd, you can create a file in /etc/modprobe.d/ with the following details to do the same.

blacklist

 

Next we move on to CPU related stuff, install Thermald from your package manager & enable it on boot time.

Thermald is a Linux daemon used to prevent the overheating of platforms. This daemon monitors temperature and applies compensation using available cooling methods. You can find further information on cpu power saving on the wiki: https://wiki.archlinux.org/index.php/CPU_frequency_scaling

 

Also laptops these days come up with different types of discrete cards from Nvidia to Radeon & the drivers you decide to use will also impact your battery life. Personally for me I find the use of discrete cards like Radeon unnecessary as I don’t do gaming on laptops. So I’ve also disabled the discrete card at boot time, running only the internal intel card to save power and it reduces the heat aswell keeping your laptop cool at all times. You can do the same by following this guide: https://wiki.archlinux.org/index.php/hybrid_graphics#Fully_Power_Down_Discrete_GPU

 

Finally, there’s this full run down of everything that I’ve covered in Arch wiki which you can use as a reference: https://wiki.archlinux.org/index.php/Power_saving

 

Personally, after applying all the configuration changes the power usage on my laptop dropped from somewhere around 16-20W to about 11-14W. That my friend is a major drop in power usage. Usually I used to have a discrete card running at all times & due to lack of a working driver (new card) it used to run in the background without even being used resulting in such high power usage. Now I’ve finally been able to power down the GPU & thus my battery life has been extended from somewhere around 1-1.5 hr to 2.5-3 hours and best of all it’s cool at all times.

 

In the next part we discuss in details some of the advanced aspects of power saving I mentioned earlier & it can also boost your battery life quite significantly depending on your hardware, etc.

Manjaro Linux Review

I have distro hopped like a grass hopper in my early years as a Linux user. I’ve tried different variants of Linux & one of the things that I’ve realised is that even though overall eye candy does matter, but what matters more in the longterm is your hardware compatibility, familiarity with the distro and how well it suits the flow of your work.

They say if it works, don’t try to fix it & thus we have distros like Arch and Gentoo. Gentoo is the type of distro that not only requires pure commitment and dedication, but you also need a lot of patience to go through hours of frustrating compilation process. The end result being that all your software is optimised according to your architecture; in non-geek terms it means that your computer would pretty much run the fastest it can run and more. It’s like replacing the Toyota engine of your car with that of a Ferrari. Who wouldn’t want the extra speed right? Only if you knew how painful the actual process is.

Anyway, Arch on the other hand has prebuilt binaries optimised for your architecture which is similar to that of Gentoo but the upside being that there’s no compilation process therefore it’s much faster to get things setup. With Arch you’ll have to start from scratch and go through a big book full of instructions on how to set it up and running according to your taste. Nothing impossible, purely do-able but you need the time & energy to troubleshoot your own hardware, software & driver issues along the way. Don’t get me wrong, I love these distros because they give you total control over how you want your perfect system to be like but I’m certainly not the kind of guy that’d wait 10+ hours just to get my system to a usable state. Therefore we have a distro called Manjaro Linux on for review today!

First of all, I’m a big fan of openbox window manager. It’s sleek, fast & totally customisable. Also I do like Arch but due to time constraint never really bothered to give it a full try.

What Manjaro Linux brings to the table is that its based on Arch Linux but it comes out of the box as a minimal install (not as bloated as Ubuntu) on which you can further customise and tune according to your taste. It brings all the goodies of arch like the keep it simple philosophy, consistent & simple bloat free architecture and most important of all user friendliness with a lot of built in scripts to assist the user in customising look & feel. So basically you get an Arch system for free without having to break a sweat.

Manjaro Linux comes in various flavors like Openbox, Kde, Mate, etc. but for now I’m going to keep this review relevant only to Openbox. The best openbox distro that I’ve used in the past is Crunchbang, and it’s really good. But what Manjaro Linux gives you is access to more up to date, bleeding edge software, increased computer performance, ability to update & switch kernels on the fly (built in scripts) & the impressive amount of documentation. I was also pleasantly surprised to see a step by step instruction manual regarding the process of installation on Manjaro (in the form of pdf), detailed with screenshots. Seems like they’ve really put up a lot of effort in trying to make it look more professional and easy to use.

TLDR; here are the things that I like about Manjaro Linux:

– Fast, simple & based on Arch Linux.

– Stable repositories. You gain access to Arch repos & a lot of tests are done to ensure that the update doesn’t break anything before it’s released on Manjaro servers.

– Built in scripts like mhwd, mhwd-gpu, mhwd-kernel ensure that you’re uptodate with the latest kernel or drivers for your graphics card & you can opt to change to different versions as you see fit.

– Openbox install comes preinstalled with all the necessary scripts, themes & customisations to just get you started without starting from a plain vanilla install.

– Big community of active users who are constantly working on fixing old scripts, creating new ones & making your Manjaro better.

– Good forum support for Manjaro users via their official forum. Documentation from Arch can also be used as a reference material to troubleshoot your problem as there’s similarity between both distros.

Things that I’d like to see improved in Manjaro:

– Video tutorials series detailing how to get certain things done as Manjaro has a strong focus on beginner Linux users.

– More customisation options for Openbox like which screensaver, login manager, etc. you’d like to install. Also it’d be nice if there’s an option to turn on some composition (compton) with fade effect & transparency by default.

– More marketing to get the message across about Manjaro, it doesn’t get as much publicity as let’s say Ubuntu.

Here’s a link to my new Manjaro install, see how it looks like:

http://irenicus09.deviantart.com/art/Manjaro-Linux-437550988

Overall, I’m really impressed with Manjaro Linux & I’ve never seen a distro so focused towards beginner Linux users. They’ve done a great job at it & if I were to recommend a distro to my friends or family this would be it. Certainly worth giving a try if you haven’t already 🙂

Reason why Linux is not main stream yet…

Alright, I wanted to talk about a lot of the things that have been bugging me lately. This post is totally my personal opinion and is based off on my experience with GNU/Linux.

First of all, let me tell you that I’ve personally tried various distributions (distros) of Linux…from Ubuntu to Gentoo, Fedora, Arch and every thing in between but I don’t consider myself an expert. This post highlights what I think is wrong with the GNU/Linux in general and what can be improved to make it more appealing to the end users.

The various distributions of Linux that I’ve tried are quite similar in nature, they have a lot of the commonalities like the kernel, the architecture and software that run on it but they vary based on the philosophy among their communities, their vision, their goals, their package managers, etc.

For example Gentoo’s philosophy is that it’s a source based distribution where you have to compile everything from source where as others like Ubuntu rely mainly on binary builds.

Anyway that’s not what I wanted to talk about. What I’m trying to highlight here is that a lot of the things that we do in the Linux world is just too damn complicated for a typical end user to go through.

For example, I bought a new laptop eight months ago and it came with a proprietary AMD Radeon HD 8750m card with dual switchable graphics. When I bought it I didn’t expect the card to work at all since it’s so new. But recently I realised that the support for this card is available in the newer 3.11 kernel which allows dynamic power management to improve battery life and decrease heating, etc.

Therefore I decided to install the latest kernel for which I had to use a script (smxi on Debian) to get it installed. Even after installing the kernel, next issue was that I had to enable dynamic power management (dpm) in the boot settings (grub) manually as it was disabled by default. Then again it didn’t work because I was missing some firmware or because I had to disable the proprietary intel driver and load the radeon driver manually. Or may be because of something else that I need to spend time experimenting.

My point is that a typical end user shouldn’t be worrying about these kinds of stuff. In the Linux world we expect the user to be able to manage and handle these kinds of stuff. We assume they have some knowledge of bash, terminal navigation or even programming skills.

In contrast, the Windows or MacOSX world is like handing the end users freshly baked cakes out of the oven. They don’t need to know anything about baking power or the ingredients that goes into it, they don’t need to worry about the correct temperature in the oven….they just enjoy their freshly baked cakes.

That is the reason why I think people (general population) are unlikely to come over to Linux anytime in the future except for hobbyists, geeks and people who need to run Linux for their day jobs. But I can not totally disregard the effort distros like Ubuntu have been putting in….to make a big impact in the department of user friendliness, making all these subtle things that is so hard for a normal Windows user….fade away. So I should obviously give props to them for trying to make Linux go in that direction.

But still, from my last experience with Ubuntu 12.04 there’s a lot of things I didn’t like….let me start off with the Zeitgist and online search bullshit. Whenever you search on the dash, your search results get sent off directly to Ubuntu servers and that is turned on by default! Talk about privacy…..next thing they had was this Geo-location service thing going on and from what I’ve read on Ubuntu forums itself…when your computer boots off it tries to ping Ubuntu server or something similar even while you’re not logged in. So ya that totally turned me off.

Anyway there’s still hope that Linux can make it to the main stream in the Desktop world if SteamOS can be a big hit. Valve is a pretty resourceful, reputable company and the steam platform itself has hundreds and thousand of AAA titles. If they can somehow make all these subtle things in Linux that we have to deal with every day go away and make Linux more usable then I think people can be convinced to switch over to Linux after all.

Don’t get me wrong, I love Linux and use it on a daily basis….but I simply think that it’s not for everyone yet. At my University, I even opted for teaching a Linux course as a term project in order to get more people interested in Linux. From that teaching experience, although there were a handful of students…..what impressed me the most was that even people with no prior experience with Linux were able to grasp things so easily and fast. If I imagine my personal experience with Linux itself…it took me a whole lot longer than that to get the hang of things. Perhaps it was because they had programming experience or IT background etc.  but as of yet I can not confidently go to someone who has zero knowledge of Linux and recommend that person to switch over to Linux. In my opinion they’re better off with their freshly baked cakes.

Gen2k – Automated Wordlist Generator

So, I’ve decided to brush up on my python skills and make something useful.

Enter: Gen2k

gen2k

What is it?

It’s an automated word list generator.

What is a word list?

Word list is like a list of possible passwords that you can use to crack hashes with, perform brute force attacks on various protocols & may be used for just about any cracking in general.

Why would I need an automated word list generator?

Well, actually you don’t need to generate your own as there are already some pretty good word lists floating around on the web. But there are times when you would want a personalized word list to fine tune your attacks; especially if you know the target well. In such cases automated word list generators may come in handy as it allows you to make educated guesses regarding what the password might be rather than just brute forcing with totally random, irrelevant word list.

How is it different?

Gen2k is still in beta, but works flawlessly as of now. It’s not your typical word list generator, and doesn’t intend to be one. There are already good ones out there like Crunch, etc.

Gen2k aims to be a smart word list generator, it takes sample words as input. Sample words can be anything that you know about the target, from area, date of birth to names & special events, etc. Once a list of all those known words have been supplied to Gen2k, it automatically, based on the options set..determines the best possible way to make a word list out of those. As many of you know, people tend to use birth year, specific dates, random numbers, custom words attached to simple words in order to make their passwords more complex. Gen2k aims to exploit those types of weaknesses along with conversion of words to upper & lower cases to make your word list completely personalized & appropriate for the situation.

It has most of the features that I thought of implementing when I started working on it and obviously it can be improved further. It’s written completely in Python. It’s fast, light weight & doesn’t have any external dependencies.

What are it’s features?

* Generates password combinations by combining supplied words.
* Mixes frequently used number patterns with words.
* Generates password combinations using year/date combo.
* Mixes custom user defined value(s) combination with words.
* Option to auto convert words to upper/lowercase & capitalisation.
* WPA/WPA2 password validation check.
* No external dependencies.

So what does it look like?

gen2k

The list can get very large indeed, so make sure you choose the options wisely.

Where can I get it?

Gen2k-git

Want more features? Found a serious bug that needs fixed? Need more updates?

Feel free to comment below & let me know! 🙂

Openbox Review

For quite a long time I was looking for a light weight window manager or even a full featured desktop environment to switch to from Gnome 3. Personally, I have nothing against Gnome 3…after having used it for quite a while I do see where it´s headed and I do think it´s going in the right direction with user friendliness, innovation, etc. but as a Linux user and having used Linux for a few years now I feel like all of these features are just too much for me, perhaps even unnecessary. Not to mention that my old laptop which I currently seem to use as my production machine seems to be limited by it´s ram (1 GB) and running gnome3 on top of that doesn´t leave with much resources to use with other applications if need be.

Therefore, a few days back I decided to give Openbox a try and take a break from Gnome-3, and I must say it was worth it! 🙂

First of all, I started with reading documentation related to Openbox, directly from Openbox Wiki & Gentoo Wiki on Openbox…and for the most part they have an in-depth guide that pretty much allows you to configure everything related to Openbox.

After having used Openbox, I found it to be really simple, flexible, customisable, lightweight and obviously fast! You can pretty much feel the difference in speed between Gnome3 vs Openbox, itś like comparing Toyota vs Lamborghini 😛

Anyway I don´t wish to bash Gnome-3 anymore, it does deserve credit for whatever it´s trying to achieve with user friendliness and I think it is a necessity.

As for me, I´m now a permanent Openbox fan…checkout the screenshot 🙂

Link @full size: http://i.minus.com/iRt74ggjdopjJ.png

So what do you guys think? Feel free to leave your suggestions, comments behind! Thanks 🙂

Laptop power saving on Linux

Hi guys, recently after coming back to my country over summer holidays I’ve been really frustrated with all the daily power-cuts (yes, I’m from a third world country).

So I decided to research some cool tools for Linux that would help conserve power on my laptop.

First of all, how is power usually conserved on a laptop? Without the use of utilities, if we use logic it’s pretty obvious…we can manually lower the screen’s brightness and set it to a minimum at the same time we can turn off audio completely and I think that would help a lot in conserving power. But beyond that what other stuff can be done to conserve power on our Linux system that perhaps we are not aware of? Well that’s one of the reasons these power saving utilities exist, so let’s investigate.

Today I’ll discuss my personal favorite power saving utility, it’s called “laptop-mode“. The name fits it’s purpose right? 🙂

It is a commandline based utility and the best thing about this utility is that it takes care of handling everything on the fly…everything from auto disabling unused USB devices/ports to scaling down CPU frequency…to lowering hard-drive spinning rates, etc. is dynamically adjusted when you plug in/out of power.

Also it comes with a really well documented, simple configuration file located in “/etc/laptop-mode/laptop-mode.conf” which you can change to adjust how you want to manage your power.

Usually, before using laptop-mode I had approximately 1.5 hrs  of battery life on my old laptop provided brightness & sound was set to a minimum. But after using laptop-mode & under the same condition it seems that my laptop power lasts for approximately 2.5 hours…an hour more saved! That is something isn’t it? 🙂

So how can you get your hands on laptop-mode? Search for it in your package manager and I’m pretty sure it is available, if not then perhaps you might look for it’s source & compile it manually.

As for me, everything is available on the land of Gentoo:

Well how does it save power you may ask? Well just look at the list below:

* ac97-powersave
* cpufreq
* dpms-standby
* eee-superhe
* ethernet
* exec-commands
* hal-polling
* hdparm
* intel-hda-powersave
* intel-sata-powermgmt
* nmi-watchdog
* runtime-pm
* sched-mc-power-savings
* sched-smt-power-savings
* terminal-blanking
* usb-autosuspend
* wireless-ipw-power
* wireless-iwl-power
* wireless-power

That list has been taken directly from my configuration and perhaps it is generated according to my laptop’s hardware.

Another good utility to look out for is “powertop“. It’s similar to laptop-mode with similar features but the only disadvantage that I can think of is that it doesn’t dynamically switch between power saving modes or remember your power saving settings so you may need to manually enable/disable various options from it’s commandline, menu driven program which I think is quite annoying.

Anyway that’s about it from me, hope this post has helped you in your quest for conserving power!

Also please leave some feedback if this post was useful to you and do let me know if you know of any better ways to save power.

Commandline Based Music Player

Sup guys,

Quite recently I was having a lot of issues with my favorite music player – Banshee, it’s kind of laggy on my underpowered laptop but also all of a sudden there’s this bug that makes the GUI interface totally unresponsive so I decided to look at other options.

All the other options like Rhythmbox, Amarok, Audacious for some reason didn’t quite fit what I was looking for so I decided to research for a commandline based Music player & only then I stumbled upon CMUS.

http://cmus.sourceforge.net/

This is how it looks like:

The main thing I was looking for in a music player was that it had to use minimal resources & CMUS does exactly that since it’s terminal based & has no GUI interface. It utilizes ncurses library to interact with user & I think it’s quite user friendly too, it’s quite similar to IRSSI (cmdline based IRC client).

But the main feature of CMUS is that it’s quite customisable and it’s configurations can be easily edited similar to the cmdline based editor Vim. It has all the features I was looking for from a music player like Banshee & interestingly there’s also a plugin called last-cmus that allows you to scrobble tracks to lastfm – http://code.google.com/p/last-cmus/

So far I’m quite satisfied with CMUS and I don’t think I’ll be switching to anything else anytime soon.

Anyway, if you are a CMUS user feel free to share what you think about it! 🙂

Linux security guide Part-1

Hi guys, I’ve decided to write up a brief series of article on how you can monitor your Linux box and check whether it has been compromised. This guide will be brief, straight to the point and geared towards beginner Linux users. It will also state some of the best practices and some good prevention methods that can be used to reduce the security risk of your Linux box.

Note:  This guide will be generic and applicable to all Linux distros. I will not bother going into the details on how you can install a particular utility for your distro since there are probably a thousand distros out there. Also the commands that are meant to be executed in terminal are written in bold letters and enclosed in quotes.

First of all security is a pretty broad term, and there’s many aspects of it…so it’s not practical enough for me to cover everything in a single guide. Thus I’ll just focus on the important parts and assume that you are at-least comfortable with the command-line and know the basics of Linux. So, let’s get started.

Network:

So first of all we need to monitor our network since from a security perspective, it’s the first thing that comes to mind when we think about a computer connected to the Internet. We have several command-line utilities that can be used, I’ll just stick to some of my favorites for now.

1) Closing unused services/ports and scanning with Nmap

Nmap is an excellent tool to scan our own computer. We can see what ports and services are open, and whether there are any back-doors listening for remote connections. Install nmap from your package manager, and keep reading.

We can do a basic scan on our own computer by typing the following commands below.

To scan for open ports type: “sudo nmap -sS -p 1-65535 -v localhost

To scan for services running on open ports, type: “sudo  nmap -A -p 1-65535 -v localhost

The commands above scans for open ports and services, you should see something like this:

Note that in my case there are two open ports: 631 and 10,000.

The service ipp is “Internet Printing Protocol” which is used for printing related tasks; I can leave that open since occasionally I do use a printer, etc.

As for port 10,000 it’s a firewall (Shorewall Firewall) running with a front-end called “Web-min”. Webmin and shorewall firewall are great combination that may be you should check out, but for now we’ll just skip that part.

Once you’re done with the scan, you may find a lot of services like cups, avahi-daemon, etc. running on your system. There’s no need to start panicking right now, since some of them are usually installed by default on distros like Ubuntu. If you’re not using a particular service, I would definitely recommend that you remove it since it may pose a security risk in the future.

Consult your distro’s documentation or do a quick search on Google to find out how you can add/remove services specific to your distro. Again for knowing what a particular service is related to and whether you should consider removing it, Google is your best friend.

Let me give an example related to  Sabayon Linux (Gentoo based distro):

To remove a service: “rc-update del <service name> default

To add a new service: “rc-update add <service name> default

For example if you want to remove avahi-daemon, you can type: “rc-update del avahi-daemon default

Also in case you want to directly add/remove a particular service you can do something like this:

# Command below lists the scripts/services that are available, not necessarily active

ls /etc/init.d/

# The command below stops a particular service – cupsd

/etc/init.d/cupsd stop

Note that the “stop” keyword stops the service you can also use other keywords like “status” or “start” to check the status and start the service respectively.

Obviously I expect you to have root level priviledges to successfully carry out the commands.

Closing unused services and open-ports can dramatically reduce the security risk of your Linux box. The more services and open ports you have running, the greater the risk of a security breach.

This concludes the tutorial for today, and I have decided to write the entire guide in parts since I don’t have the time to write everything in a day. If this guide interests you then consider subscribing to my blog through rss or email, which ever is convenient. The next part will focus on network (we still have a lot to talk about) and I’ll discuss other network monitoring tools, so see you soon!

Cooking with Beautiful Soup

Hey guys, today I’ll be showing you a simple example on how to use Beautiful Soup to scrape data. To be more specific we will be looking at an open public forum and just extract all the thread’s title on the first page followed by some meta-data like author, post date, etc.

What is BeautifulSoup?

Well in simple words it’s a Python module that can be used for webscraping, data Mining, and for all your parsing needs.

Why should I consider using it?

To be honest there are a lot of parsers out there for Python…lxml, SGML, etc. but the main feature of Beautiful Soup is that  it’s quite easy to use and developer friendly; you can easily construct a few expressions and with just a few lines of code you can get your job done. Aim of Beautiful Soup is mainly to be developer friendly, but performance wise may be it’s not the best. If performance is your main aim, then other C based parsers like lxml might fit your bill.

Anyway let’s proceed with using Beautiful Soup!

The script that I’ve released here is coded in such a way so that it’s not intrusive, and only makes a single request just like any other user who is viewing the forum.

First of all you will need Python 2.7 or earlier installed along with the appropriate version of “Beautiful Soup”.

View/Download the script from here –> http://pastebin.com/3wrnDw12

Link to the forum we want to scrape –> http://forum.intern0t.net/perl-python/

As you can see the code is pretty simple and self explanatory. Beautiful Soup makes it quite easy to refer to different types of tags using various attribute values and we can easily play around with different nodes to get to the data we want.

If you run the script this is what you’ll see:

See how simple it is? It only took 5-6 lines of code related to Beautiful Soup to achieve that. This is something that made me fall in love Python; it’s simply too elegant and simple.

In the past before I knew Beautiful Soup, I used regular expressions mainly to get the job done. Regular expressions are great for complex tasks, you can even use Regex with in Beautiful Soup itself if you feel like doing so.

Anyway for those of you who are new to Beautiful Soup I would recommend playing around with the code and learning it through your personal experience.

Don’t forget to look at the Official Beautiful Soup Documentation as reference –> http://www.crummy.com/software/BeautifulSoup/documentation.html

Goodluck!