rsnapshot: Auto Version Controlled Backup for Unix/Linux/Mac/BSD…

Red Had Enterprise Linux (RHEL) 6. I missed the ease of configuration and all the free tools that people smarter than me have created.

Systems that can take advantage of Rsnapshot

Systems that can take advantage of Rsnapshot

I would like to do a fast post on rsnapshot. I have seen ssh and rsnapshot scheduled in cron to automate backups of OSX to a Linux server. Since we didn’t want the wireless to slow down we only used the physical MAC address of the MAC. What makes rsnapshot so great is that it will wok on so many systems that are out there (Ubuntu, Debian GNU/Linux, Red Hat Linux, Fedora Linux, SuSE Linux, Gentoo Linux, Slackware Linux, FreeBSD, OpenBSD, NetBSD, Solaris, Mac OS X, and even IRIX) .

For now I’m using it for personal automated backups to my external hard drive. There are plenty of other advanced options and examples on the Internet. I just wanted to get out a fast an easy example.

  1. First – find and install rsnapshot. for Red Hat this was
    $ sudo yum install rsnapshot
    (rsynch is a dependancy that should already be installed).
  2. After install if you do not have this file /etc/rsnapshot.conf. Use the command:
    $ sudo cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf
  3. Edit rsnapshot.conf – The defaults I changed from the default configuration file are below. These options allow me to back up everything in /etc/ and /home/. Backups kept will be twice a day, 7 days a week, 4 weeks, 12 months and 5 years (change this as you see fit).  Most important is that switch to make sure that the mount point will not be created and wrote to locally if the disk is not attached.
    1. WHERE TO PLACE BACKUPS
      # All snapshots will be stored under this root directory.
      #
      snapshot_root   /media/myexternal/rsnapshot/
    2. DO NOT CREATE IF DISK IS NOT CREATED
      # If no_create_root is enabled, rsnapshot will not automatically create the
      # snapshot_root directory. This is particularly useful if you are backing
      # up to removable media, such as a FireWire or USB drive.
      #
      no_create_root 1
    3. INTERVALS (make sure this is tabbed – do NOT use spaces)
      #########################################
      #           BACKUP INTERVALS            #
      # Must be unique and in ascending order #
      # i.e. hourly, daily, weekly, etc.      #
      #########################################
      interval        hourly  12
      interval        daily   7
      interval        weekly  4
      interval        monthly 12
      interval        yearly  5
  4. Time to configure cron. Most people will tell you to create your jobs using $ crontab e
    I prefer to use the root crontab using $ sudo vim /etc/crontab shown below:

    0 */12 * * * root /usr/bin/rsnapshot hourly # Every 12 hours
    30 23 * * * toot /usr/bin/rsnapshot daily   # Daily at 11:30PM
    20 2 * * 0 root /usr/bin/rsnapshot weekly   # Sunday at 2:20AM
    10 5 1 * * root /usr/bin/rsnapshot monthly  # First day of the month at 5:10AM
    01 8 1 1 * root /usr/bin/rsnapshot yearly   # January 1st at 8:01AM
  5. Test It – Following these steps you should have the basic setup needed to run rsnapshot on your personal computer to an external hard drive or usb. Just one last thing to do. Make sure that your hard drive is plugged in and  run:
    $ sudo rsnapshot -V hourly
    rsnapshot should give you plenty of verbose information as it creates your first hourly backup inside the location you specified. If there is a issue with the lock file, remove the lock file and try again.

Still stuck?

There are many other helpful documents out there  start with the rsnapshot how to:
http://www.rsnapshot.org/howto/1.2/rsnapshot-HOWTO.en.html#installation

If you want to learn how to do remote backup and use OSX? try this article:
http://blog.philippmetzler.com/?p=138

As I said in the beginning of this article, this was a fastpost and not meant to cove everything about rsnapshot. It took longer to write this article than it did to set up rsnapshot.
Good Luck – Adam M. Erickson

 

[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

Expect – For more remote work, automation, & CI

If you ever have a situation were you want to run scripts automatically this will help.
You can even time the events through cron or CI (continual integration) software like Hudson/Jenkins.
Let me introduce you to a new command I was not aware of until this year.

It is called Expect.expect

Expect can be installed like any other package in a Linux environment.

  • $ sudo yum install expect
  • $ sudo apt-get install expect

Here is an example of what created inside Jenkins to run after a build as an shell script.

Instead of #!/usr/bin/bash

Use:hudson

#!/usr/bin/expect
set prompt "$ "
spawn ssh -t user@remote.server
set timeout 30
expect {
timeout {
puts "Connection timed out"
exit 1
}
"yes/no" {
send "yes\r"
exp_continue
}
"assword:" {
send -- "job00ibm\r"
exp_continue
}
"$prompt" {
send "cd ~/test/\r"
}
}

Now, as long as you have Internet, the correct username, remote server, and password. You should of just created a script the will SSH into a remote server and accept the RSA fingerprint. Or you just created a script that tells you the connection timed out.
Don’t alter this part of the script much because you want to be able to re-use and if there is not a yes/no at the prompt it will continue to expect assword:.
You can get change the last command to anything you want. I added that part because you remotes into a server for no reason?

Add this to the script and it will send the command to find anything with a certain name in it to be removed from the current directory.
expect {
"$prompt" {
send "find . -name '*filename*' -type f -print0 |xargs -0 rm -f\r"
}
}

Add this to the script to find files of a certain name and copy to a new location. You should be able to use modified times in the find switch if that helps.
expect {
"$prompt" {
send "find . -name 'coke*' -type f | xargs -n1 -i cp {} /data02/home/ibmcorp/12180-us-mcrtest1/upload/\r"
}
}

Add this to the script to run a script in the current directory.
expect {
"$prompt" {
send ". name.sh\r"
}

Does your script prompt for anything? Expect can enter the reply for you.
}
expect {
"the expected promt" {
send -- "the reply\r"
}
}

Some things need to e escaped like these ampersands. It is not way to find out what all needs to be escaped but you can figure that through trial and error, intuition, or because of how smart you are.
expect {
":" {
send "me\@a-erickson.com\r"
}
}

That’s my real address if you want to reach me.

All done running your script? Don’t forget to exit your connection.
expect {
"$ " {
send "exit\r"
}
}remote

  Have fun work remotely 🙂

Develope a Web Based CMS Using PHP

Download this File Here

Abstract
The Content Management System (CMS) is a web based application using a Linux Server,
Apache Web-server, MySQL Database, and PHP Programming Language (LAMP). The
objective of managing users, and information in any given network environment can only be
hindered by the creativity of an information technology professional and not by technology. The
main objective of this thesis is to develop the early development steps of a LAMP software bundleCMS. By creating the
building blocks for developing, and taking into consideration basic methods for creating the core
platform of a CMS for further development. All information gathered, and experience gained will
assist with developing and offering my own personal e-commerce business solutions in the future
and to obtain additional business and practical knowledge in an open source software and ecommerce.
Continue reading

Enhance VIM

While VI has been used for years, in current Linus & iOS systems a lot of people have resorted to using Vim.

There are a lot of tricks to Vim that can make it easier to use. Some may not be active by default and I wanted a chance to introduce some of them to you. One in general that I have found very useful.

set background=dark

Lets start by making sure that you have the packaged “vim-enhanced” installed on your system. Use one of these steps from a terminal prompt to install it.

Mint/Ubuntu (Debian) sudo apt-get install vim-enhanced

Fedora (redhat, centOS) sudo yum -y install vim-enhanced

OpenSuse (Suse) sudo zypper install  vim-enhanced 

If one of these does not work or your operating systems is different, there are several other options for installing packages. Try the GUI option as well.

The best way to modify settings in Vim is to have the .vimrc file (Vim Run Command file) in your home directory.

cp etc/vimrc  ~/.vimrc

Open the file,

vim ~/.vimrc

From here you can fix all sorts of text editor effects in Vim.

Just remove the ” from the comment in order to use it.

The one that made a difference for me was.

set background=dark

This is a GREAT setting if you are working in Vim and have a dark background.

It will change what you see from this:

Vim background not set.

“set background=dark

To this:

Vim

set background=dark

set background=dark

Making it very easy to see commenting, now there are several ways to change the config file and in the last image you can read some of the ones in the file we copied (cp) over.

Besides being able to read things better know I also like it when the cursor is in t he same place from the last time I was in the file. Another on on the top of the list is the ability to scroll the cursor with me mouse.

Play around with the file, you can always delete it and start from the beginning again if you mess it up to bad.[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

Open Source Software in Digital Forensics

The purpose of this research paper is to research information on open source digital forensic tools that are assess-able for free, usually online. To review types of digital forensic tools available and what they do. The basic definition of what open source and digital forensics is will be defined, and how Open Source Software (OSS) digital forensic tools can help accomplished data retrieval. The pros and cons of why OSS should be considering as a viable digital forensic tool-set is also covered.

Digital Forensics and Incident Response and Tools

Digital forensics and Incident Response (DFIR) is the method of investigating and analyzing data information for the purpose of presenting, an ordered report that shows a chain of evidence to find out what happened on a computer and who was responsible, to a court of law. SearchSecurity. (September 2004). DFIR is being more commonly used as more and more people use computers in their daily life, from smart phones, game stations, and laptops. DFIR can help convict anyone of any crime that involved a computer, wither it is prostitution, child pornography or a white collar crime like embezzlement.

DFIR Tools are the free and proprietary applications used by DFIR experts to retrieve the results to hand over to the legal system. They allow investigators the ability to examine the contents of the hard drive without making changes to the data held within. Information that is retrieved can come from deleted files, encrypted, or damaged files SearchSecurity. (September 2004).

Open Source Software

Open Source

Some things are priceless.

Open Source Software (OSS) is a set of practices used to collaborate with software source code that has been made freely available through copywriting laws. It is also commonly known as FOSS (Free Open Source Software), although most OSS is free, not all is but for this research paper I will be covering mostly the free version of OSS. Individuals separated can come from diverse cultural, corporate boundaries, language and other characteristics in order to work together to create complex, non-proprietary software. Software is open sourced when it is free to redistribute, the source code is redistributed with it as well as in compiled mode. The open source licensing was created to make the source code of a program readily available to anyone that requests it. By making the source code available for anyone, it helps in developing stable software because the whole community is able to create changes and redistribute their own version of the software. Open source software protects the original author of the software, does not discriminate in anyway on how it can be used, cannot be specific to a product or software, cannot restrict other software and has to be technology neutral (open source, n.d.). There are several variants of the open source licensing contract that can be reviewed at opensource.org (http://www.opensource.org/licenses/category).

Some of the more widely known open source licenses cover the GNU(Graphic environment of sever Linux desktops), Mozilla (Firefox, Thunderbird), MIT, BSD (like Unix), and Eclipse (Eclipse IDE). Because of the lack of dependency on software vendors, open source software allows the software to transform and morph into potentially anything the users and developers need the software to do. It gives users the freedom to use it when they want, how they want and on their own terms.

Why OSS DFIR Tools

Open Source digital forensic tools addresses specific gap in forensic capabilities of proprietary DFIR tools.  The tools range for analyzing memory dumps, disks, network traces, cell phones, and memory images from game consoles. Besides the fact that some of the tools focus on one specific area of digital forensics of the incident response building, make them invaluable to some investigators who find the complete packaging of some DFIR proprietary tools cumbersome and lacking in some areas.

Financially, companies and governments are always looking for ways to cut budget costs that is the same for DFIR investigators trying to find work. An investigator using DFIR tools can offer a lower price to customers than a person that has to forward the expensive cost of proprietary tools to the customer. Even law enforcement that has an easier time justifying expenses to budget reports in other departments like traffic enforcement, and drug trafficking. Because of the high cost involved in proprietary applications, the follow up cost for updates may be neglected, leaving the software antiquated and not viable in future investigations.

Legally procedures for finding digital evidence need to be defended in court as being testable, published under peer review, show the possible error rate, and are marginally accepted in the relevant scientific community. Because proprietary tools are closed source and the companies offering the tools do not what to acknowledge mistakes in their software, it makes a case all by itself for choosing OSS digital forensic tools during investigation. OSS allows the source code to be evaluated, tested and error rates to be traced. OSS tools are also greatly accepted by the DFIR community (Carrier, 2002). As Brian Carrier, (Carrier, 2002) reported, “The digital forensic market should not be approached in the same way that other software markets are. The goal of a digital forensic tool should not be market domination by keeping procedural techniques secret.” While Carrier may be a little bias since he developed most of the code in Sleuth Kit, Autopsy, and mac-robber, his experience in digital forensic just proves the fact that it is important to keep OSS DGIR tools in mind.

Counterproductive to this paper, it is important to note that there usually is a larger learning curve when dealing with OSS DFIR tools since some run from command prompts and from Nix (Linux, Unix, BSD) operating systems. Also since they usually focus on one component of DFIR, it requires several different applications that need to be tied together to build a report. Because some of the tools take a lot of time to college and arrange data for a report it is sometimes better to use in a lab then in the field.

Conclusion

While open source digital forensic  tools are abound, and you can take advantage of all of them while  avoiding paying fees for commercial products, there are several good commercial tools that are available also. Because digital forensics is such a vast field of study, it is important to not rely on just one set of tools and to research and test other methods to discover and fight anti-forensics.

Good luck with any future digital forensic test cases you attempt, please make sure that it is done ethically and legally.

More information on OSS DFIR tools can be found at sites like IEEE, open source references at the National Institute of Technology (NIST), the National Software Reference Library (NSRL) form NIST, and government studies and college studies on OSS alternatives in DFIR. Additionally there is a growing amount of information from personal websites and OSS developers on DFIR that may be useful. I totally agree with Schneier (2010), that we, “would encorage everybody to download and learn the tools not just because they can do forensics but because most of them can also be used for other things such as finding things in memory and hard drives that should not be there which many AV tools cannot do and to help put systems back together again.”


Future Note: I plan to compare and contrast some of the more common tools that I will continue to study.References

References

Carrier B, (2012). Slueth Kit. Retrieved 4/15/2012. From http://www.sleuthkit.org/sleuthkit/index.php

Cmihai. (October 2007). UNIX System Administration: Solaris, AIX, HP-UX, Tru64, BSD. Retrieved 4/27/2012, from http://blog.boreas.ro/2007/10/digital-forensic-tools-imaging.html

DFF (n.d.) Open Source Digital Investigation Framework. Retrieved 4/17/2012, from http://www.digital-forensic.org/

Forristal, J., Shipley G.. January 8, 2001. Vulnerability Assessment Scanners. Network Computing. http://www.nwc.com

LinuxLinks (n.d.) 6 of the Best Free Linux Digital Forensics Tools, retrieved 4/5/201. From http://www.linuxlinks.com/article/20110115103656314/DigitalForensics.html

Nikkel, B. (June 2012) Practical Computer Forensics using Open Source tools, retrieved 4/23/2012, from www.digitalforensics.ch/nikkel08.pdf

Open Source. (n.d.) The Open Source Definition. Open Source Initiative. Retrieved from http://www.opensource.org/docs/osd

Open Source Digital Forensics (n.d.) Tools, retrieved 4/5/2012 

Schneier B.. (December 2010), Open Source Digital Forensics. Retrieved 4/6/2012. From

SearchSecurity. (September 2004). computer forensics (cyberforensics). Retrieved 5/4/2012. From http://searchsecurity.techtarget.com/definition/computer-forensics

[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]