rsnapshot: Auto Version Controlled Backup for Unix/Linux/Mac/BSD…

Red Had Enterprise Linux (RHEL) 6. I missed the ease of configuration and all the free tools that people smarter than me have created.

Systems that can take advantage of Rsnapshot

Systems that can take advantage of Rsnapshot

I would like to do a fast post on rsnapshot. I have seen ssh and rsnapshot scheduled in cron to automate backups of OSX to a Linux server. Since we didn’t want the wireless to slow down we only used the physical MAC address of the MAC. What makes rsnapshot so great is that it will wok on so many systems that are out there (Ubuntu, Debian GNU/Linux, Red Hat Linux, Fedora Linux, SuSE Linux, Gentoo Linux, Slackware Linux, FreeBSD, OpenBSD, NetBSD, Solaris, Mac OS X, and even IRIX) .

For now I’m using it for personal automated backups to my external hard drive. There are plenty of other advanced options and examples on the Internet. I just wanted to get out a fast an easy example.

  1. First – find and install rsnapshot. for Red Hat this was
    $ sudo yum install rsnapshot
    (rsynch is a dependancy that should already be installed).
  2. After install if you do not have this file /etc/rsnapshot.conf. Use the command:
    $ sudo cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf
  3. Edit rsnapshot.conf – The defaults I changed from the default configuration file are below. These options allow me to back up everything in /etc/ and /home/. Backups kept will be twice a day, 7 days a week, 4 weeks, 12 months and 5 years (change this as you see fit).  Most important is that switch to make sure that the mount point will not be created and wrote to locally if the disk is not attached.
      # All snapshots will be stored under this root directory.
      snapshot_root   /media/myexternal/rsnapshot/
      # If no_create_root is enabled, rsnapshot will not automatically create the
      # snapshot_root directory. This is particularly useful if you are backing
      # up to removable media, such as a FireWire or USB drive.
      no_create_root 1
    3. INTERVALS (make sure this is tabbed – do NOT use spaces)
      #           BACKUP INTERVALS            #
      # Must be unique and in ascending order #
      # i.e. hourly, daily, weekly, etc.      #
      interval        hourly  12
      interval        daily   7
      interval        weekly  4
      interval        monthly 12
      interval        yearly  5
  4. Time to configure cron. Most people will tell you to create your jobs using $ crontab e
    I prefer to use the root crontab using $ sudo vim /etc/crontab shown below:

    0 */12 * * * root /usr/bin/rsnapshot hourly # Every 12 hours
    30 23 * * * toot /usr/bin/rsnapshot daily   # Daily at 11:30PM
    20 2 * * 0 root /usr/bin/rsnapshot weekly   # Sunday at 2:20AM
    10 5 1 * * root /usr/bin/rsnapshot monthly  # First day of the month at 5:10AM
    01 8 1 1 * root /usr/bin/rsnapshot yearly   # January 1st at 8:01AM
  5. Test It – Following these steps you should have the basic setup needed to run rsnapshot on your personal computer to an external hard drive or usb. Just one last thing to do. Make sure that your hard drive is plugged in and  run:
    $ sudo rsnapshot -V hourly
    rsnapshot should give you plenty of verbose information as it creates your first hourly backup inside the location you specified. If there is a issue with the lock file, remove the lock file and try again.

Still stuck?

There are many other helpful documents out there  start with the rsnapshot how to:

If you want to learn how to do remote backup and use OSX? try this article:

As I said in the beginning of this article, this was a fastpost and not meant to cove everything about rsnapshot. It took longer to write this article than it did to set up rsnapshot.
Good Luck – Adam M. Erickson



Expect – For more remote work, automation, & CI

If you ever have a situation were you want to run scripts automatically this will help.
You can even time the events through cron or CI (continual integration) software like Hudson/Jenkins.
Let me introduce you to a new command I was not aware of until this year.

It is called Expect.expect

Expect can be installed like any other package in a Linux environment.

  • $ sudo yum install expect
  • $ sudo apt-get install expect

Here is an example of what created inside Jenkins to run after a build as an shell script.

Instead of #!/usr/bin/bash


set prompt "$ "
spawn ssh -t user@remote.server
set timeout 30
expect {
timeout {
puts "Connection timed out"
exit 1
"yes/no" {
send "yes\r"
"assword:" {
send -- "job00ibm\r"
"$prompt" {
send "cd ~/test/\r"

Now, as long as you have Internet, the correct username, remote server, and password. You should of just created a script the will SSH into a remote server and accept the RSA fingerprint. Or you just created a script that tells you the connection timed out.
Don’t alter this part of the script much because you want to be able to re-use and if there is not a yes/no at the prompt it will continue to expect assword:.
You can get change the last command to anything you want. I added that part because you remotes into a server for no reason?

Add this to the script and it will send the command to find anything with a certain name in it to be removed from the current directory.
expect {
"$prompt" {
send "find . -name '*filename*' -type f -print0 |xargs -0 rm -f\r"

Add this to the script to find files of a certain name and copy to a new location. You should be able to use modified times in the find switch if that helps.
expect {
"$prompt" {
send "find . -name 'coke*' -type f | xargs -n1 -i cp {} /data02/home/ibmcorp/12180-us-mcrtest1/upload/\r"

Add this to the script to run a script in the current directory.
expect {
"$prompt" {
send ".\r"

Does your script prompt for anything? Expect can enter the reply for you.
expect {
"the expected promt" {
send -- "the reply\r"

Some things need to e escaped like these ampersands. It is not way to find out what all needs to be escaped but you can figure that through trial and error, intuition, or because of how smart you are.
expect {
":" {
send "me\\r"

That’s my real address if you want to reach me.

All done running your script? Don’t forget to exit your connection.
expect {
"$ " {
send "exit\r"

  Have fun work remotely 🙂

Ethics in Business Intelligence


The type of ethics in business intelligence (BI) is the ethical principles of conduct that govern an individual in the workplace or a company in general.

It is also known as professional ethics and not to be confused with other forms of philosophical ethics including religious conviction, or popular conviction. Professional ethics according to Griffin (1986) is that profit is not the only important strategy of a business anymore. There is also more of a concern and motivator of companies to do what is right.

Companies must acknowledge that they have a common good to protect there local community, improve employee relations and promote informational press to the public. While back in 1986, Griffin was directing his argument towards ethics in accounting but it is also true today in Business Intelligence. Government regulations are not changing fast enough to cover all the changes in technology that bombards users on day to day bases. It is up to corporations to create a code of ethics, and to persistently be receptive to the needs of the public being served.

Everyday in BI management professionals may be at risk of making unethical practices in there decisions that regards the consumer, business and/or other employees data. Ethics is a touchy subject, there is always going to be controversy on how companies choose to handle business decisions. There is no definite decision to make when it comes to ethical decisions. While sometimes it may involve illegal practices, other times it is just a decision that needs to be made in a company to promote a better way of life for all.

An example of an ethical decision:

A manager of a BI system that chooses to use cheaper data in his/her data mining activities to save money. The data he/she chooses to implement involves personal credit score reports. The cheaper data sets have a 20% possibility of being incorrect. The manager did not see it as being an unethical decision when it was made, just a way to continue to generate close-to-accurate reports and save money.

The impacting decision on 20% of the company’s customers may have different results as more people are turned down for credit because inaccurate reports. It is not a crime to have implemented the inaccurate data sets but it may seem as an unethical practice to others.

While it is important for managers to be able to make their own decisions, this example decision being made should have involved more managers since it affected the whole business.

The manager’s choice could bankrupt the company as user start to leave their business for more accurate competitive companies. As the example points out, sometimes there is no really clear answer to wither an issue involves an ethical or legal choice and each situation can be different. Trying to make decisions based on individuals’ beliefs when dealing with a company can amount to intellectual stalls and trying to come to a decision can be expensive and time consuming.

 Business Globalization

Today’s society has come to the point where there are more solutions to problems than ever before. What once was impossible can now be accomplished through the use of BI and other technology similar to BI. It is not going to stop; technology is going to keep advancing. What seems improbable now may be common in the near future.

Because of business globalization, there is also a larger separation between companies and customers, companies and competitors than there was when everything was done locally in the past. Larger separation between companies and the consumer has resulted in unethical and sometimes illegal business decisions like data theft. Because of all the technology used in big businesses, and resulting exposure to unethical practices by some of the larger corporations like Enron, there is growing anxiety of large companies to be free of unethical practices.

Additionally the general trust level of users has eroded to the point were trust really has to be earned. Users are very aware of cases of identity information being lost to theft as well as other case examples in the media. Users have taken up with the attitude of show me or prove to me that they are safe, that there information is safe or they will not do business.

IT Personnel in Ethics

It is so easy for BI managers to sit behind there desk and manage the data on a day to day business thinking that ethical practices do not concern them. That is not the correct attitude to have. Everyone employed in the information technology field has an obligation to be part of company ethical policies and practices. It is not just about creating schemas and data models, as IT managers they have more of an ethical decision to make than there employers.

The BI manager knows more about the emerging technology, and has the best knowledge of a company’s technologies capabilities of what is possible. With all the work that is done in an informational system and what is involved in information delivery and business ethical dilemmas.

Code of Ethics

Every technologically backed association deals with ethical issues in their own way. The Association for Computing Machinery (ACM) has set some great code of ethics including:

“Computing professionals have a responsibility to share technical knowledge with the public by encouraging understanding of computing, including the impacts of computer systems and their limitations. This imperative implies an obligation to counter any false views related to computing” (ACM, 1992, para. 3).

While most of the code of ethics covers general ethical issues, it also cover leadership and other professional responsibility in information technology and is worth looking up.

PAPA Framework

PAPA is an acronym for privacy, accuracy, property, and accessibility. A framework proposed by Richard Mason as the four ethical issues of the information age.

He proposed this framework 25 years ago in 1986. To date it is still acknowledge as the four subjects of ethics in information technology and covers ethics in BI as more and more data is extracted, transformed and loaded into data warehouse silos.

A lot of are private information is handle with BI in Customer Relationship Management (CRM) systems like Amazons customer web portal. While Amazon is making web application business services for users better and geared towards individual use ,it also demands that some of your private information is given in return for the CRM to accurately predict what you may need and want.

Elements of privacy should contain a notice of what data is being collected, how it is being used, a option to participate of not, security measures to protect from data misuse, the ability to access your person information to review and correct and steps are assigned to enforce set policies. On opposing side of privacy is the need to create security, any inadequate security measures can be viewed as carelessness also while the option to participate in the data collection is an option, choosing to not participate usually means that the company will also not provide their services to you.

Accurate Data

Accuracy Data Mining (DM) and BI systems is very costly and the percentage of accurate data is a business decision. Some companies can ethically choose less accurate data and still maintain a competitive edge, and supply the users with their services while other systems like a Hospital Information System (HIS) cannot afford to reduce accuracy when a persons life in hanging on the line. When it comes down to who is responsible for the accuracy of the data, executives may set business processes for guidelines but the main responsibility stills falls to the BI manager to be able to understand their BI database and also for when new data must be integrated. Executives do not care how the analytics works, just that they are presented with accurate reports and/or dashboards.

The whole reliability and integrity of a BI system eventually is placed on the personnel who can transfers the sea of technology used, not the end users. When there is an ethical situation within the company who will be help liable, the executive who did not know the technology or the BI manager in charge of data accuracy?

Accessibility of data in the past was only privileged to a significantly smaller group of user than now. With the technology explosion of BI and web interfaces, anyone with a smart phone, computer, laptop or PDA can gain access DM information.

The technology gap, also known as the digital divide, is growing smaller.

Information is power, users have a right to be on a level playing field, we have a morale obligation to provide skills to understand and manage, understand, and access information throughout the world so that users are on a level playing field when it comes down to access of data that provides basic survival information, so a larger technology gap is not created based on poverty, sex, age, or race.

While sharing data freely is a goal to help individuals, there is a limit to what can be shared among business partners, customers and competitors yet they should also have the right to come to the same results using technology.

Ethical Issues in BI

While many ethical issue are obscure and hard to notice at the surface there is one a number one concern brought up by most users and according to Hackathorn (2005), the ethical issue in BI that is known by most is the involuntary release of personal information that has lead to identity theft.

The theft of personal information like social security numbers, birth-dates, and credit card numbers has allowed for technology skilled criminals to possibly walk away with billions of dollars in innocent victims’ money nationally.

Organization Accountability

Organization need to be accountable for financial data. The U.S. has required financial accountability through regulations like the Sarbanes-Oxley (SOX) of 2002. Yet according to Wallice (2011), the main focus of SOX is to measure internal effectiveness of business controls and does not explicitly address IT.

Because of the lack of security for IT in SOX, ISO 17700, the International Standard for the Code and Practice for Informational Security Management is being executed by companies as a framework for maintaining informational security to protect information systems from unauthorized admission, usage, modification, and destruction.

Government Use

The pressing issue of homeland security and the U.S. patriot Act after the attack on the World Trade Center in New York, left the Government with a strong ability to analyze anyone in the United States as a threat by collecting almost any type of data that they wish including financial activities and how they may be related to terrorism.

Technology is being implemented at airports in order to fight terrorism. The Transport Security Administration (TSA), according to Worthen (2006), is continuously conducting test with different data mining techniques in order to find the most effective way of weeding out terrorist so that they never gain access to be airlines again. The lack of an almost never ending budget and a lack of a well defined scope allow the TSA to try newer technologies in the name of security, compared to other sectors of business. After 9/11 the Computer Automated Passenger Pre-screening (CAPPS) system that used consumers’, names, credit card information, and address to screen for criminals was change to CAPPS II. CAPPS II combined previous technology of its predecessor with information purchased from data stores run by ChoicePoint and LexisNexis.  CAPPS was eventually replaced with a newer system called Secure Flight that shares the same process of combining passenger data with information purchased from commercial data providers. Over $125 million has been spent in the name of homeland security just in the first 5 years after 9/11.

Framework for Solving Ethical Dilemmas

The ability to solve any ethical problem is to first be aware that there is an ethical situation.

Try to be open and honest about the situation while at the same time you need to avoid discussions that could magnify the problem.

Try to make the subject of ethics in the work place an acceptable activity.

The next step is to thoroughly research the ethical problem and at the same time stay focused on the problem at hand and not try to solve the greater issues, if it is necessary for a person to solve the greater ethical issues that do not impact the company then it should be done on their own personal time. Once all research has been done on the subject and you are able to gain a better understanding to the root of the problem you need to come to a decision on what should be done to fix the ethical problem.

Once you have made the proper decision make sure that it is properly documented for you and future employees can learn from it. Solving ethical solutions is the same as solving any decision making process effectively and can be broken down into 6 simple steps: Identify the decision, get the facts, develop alternatives, rate each alternative, make the decision and implement the decision. Make sure to be clear about your actions, if you cannot come to a valuable solution on your own consider hiring someone who can.

Benefits of Ethics in IT

Employers may see that:

“Although data are mixed, numerous studies in the field of computer ethics support the hypothesis that a written and clearly transmitted code of ethics is a strong influence on employee behavior when an ethical decision is involved” (Computer Ethics, n.d.).

Companies that can change there thinking to become more ethical will also beat government regulations while implementing ethical solutions at the companies own affordable base without having to hurry up and match such regulations and will save themselves from the costs of future fines and fees for data misuses in their BI system.

If a company is well know for being able to protect the companies BI systems not only from security hacks but also from unethical practices, that company will most likely have the competitive advantage over their rivals and companies can align the business processes of their BI better to cover the broader strategy.

The main reason is to gain trust of your products and services and the ability to get a good night sleep knowing you have not cause financial or emotional harm to others.



Governments cannot change laws fast enough to protect ethical problems that are arising from new technology.

It is in the best interest of companies to be proactive when dealing with ethical situations within there companies IT department.

IT personnel do have a role to play in keeping BI systems protected and ethical.

IT personnel know the system better than anyone else in the organization and have a responsibility to help keep the data safe.

A good guideline to follow when covering ethics in data is PAPA and while you may not want to discuss ethics, a company can benefit from being ethical and choices should be made, doing nothing is always a choice but it is a poor choice when the stack of a company’s reputation is on the line.


ACM Council. (October, 1992). Code of Ethics. In Association for Computing Machinery. Retrieved April, 2011, from

Computer Ethics – Computer Ethics In The Workplace – Ethical, Companies, Company, Organizations, Norms, and Employees, (n.d.). retrieved April 25, 2011, from

Griffin, Charles H.. (1962). The Practical Philosophy of Prefessional Ethics. Journal of Accountancy (pre-1986), 113(000005), 92.  Retrieved April 24, 2011, from ABI/INFORM Global. (Document ID: 83270709).

Hackathorn, Richard. (August 2003). Ethics of Business Intelligence: A Practical Treatment Retrieved April, 2011, from

Hackathorn, Richard. (September 2005). Ethics in business intelligence. In Bolder. Retrieved April, 2011, from

Peslak, Alan R.. (2006). PAPA REVISITED: A CURRENT EMPIRICAL STUDY OF THE MASON FRAMEWORK. The Journal of Computer Information Systems, 46(3), 117-123.  Retrieved April 25, 2011, from ABI/INFORM Global. (Document ID: 1038730691).

Wallace, L., Lin, H., & Cefaratti, M.. (2011). Information Security and Sarbanes-Oxley Compliance: An Exploratory Study. Journal of Information Systems, 25(1), 185-211.  Retrieved April 25, 2011, from ABI/INFORM Global. (Document ID: 2298740021).

Digital Forensics

 forensics              Reviewing the concept of anti-forensics, which can be described as being:  “…more than technology. It is an approach to criminal hacking that can be summed up like this: Make it hard for them to find you and impossible for them to prove they found you” (Berinato, 2007).  The ultimate fear is that the rise of anti-forensics tools and techniques could make any data collected suspect, and that it jeopardizes the validity of any forensic investigation (or at least makes them so cost-prohibitive that they will seldom be feasible).  Throughout this paper we will look at what these tools and techniques are – from new developments in the field intended to conceal illegal activity to traditional anti-forensic methods to wipe data when old equipment is sold or no longer needed.  We will likewise examine the potential impact to the future of forensic investigations, as this could make the probability of a conviction extremely low.

Case studies

     After painstakingly searching several sites in the attempts to find documentation of successful anti-forensics stories and tools that were used, attempt came up pretty empty. While there are a few stories that share how people have tried to fool digital forensic experts, the fact is that no one is going to report that they were successful in fooling digital forensic investigators because they want to be able to fool them again in the future. Even the digital forensic investigators are not willing to relinquish case stories on what they found and the conclusions that they were able to come to so that they can stay an expert in their field. Some of the following stories were what I was able to find. If you ever find any interesting stories like explosives rigged into computers, or magnetic doorways, I would be interested to hear about it. Lastly I included information on how anti-forensics could be useful for personal use, in order to keep your personal information safe.

     With the amount of digital forensic cases that have been posted after the initial commencement date of this research paper, suggests that the amount of information that will be available within the next year will be an exponential growth from the amount that is available at present.

     Additionally I have come to the conclusion from reading several discussions and online expert opinions that while EnCase is the chosen digital forensic tool of use to get a broad overview of the file system, it is only one of the primary tools in an arsenal of tools that usually has a few other tools dropped into the mix and only through suggestions of peers and trial and error will you be able to decide what are the best tools for you to use.

     Just like some people to use torrents to collect illegal free music, movies, and books, pedophiles are using the same technology to spread child pornography to other pedophiles. The city of Trenton, N.J. tracked the digital fingerprints of pornographic pictures as they left one person’s computer and followed it to the next IP address and was willing to follow pictures for a total of 27 adults. One of the adults was arrested promptly before the others when officers found out he lived above a daycare facility.

     Out of the 100 state troopers and 3 months of hard work, the time came to collect the computers from the felons and extract the digital forensics necessary to convict the 27 individuals for the federal offense of either creating or having possession of child pornography. The traceable factor was the electronic watermark that was imprinted on each image. Making each image traceable on individual’s computers and also the routes the images would take on the internet. Artifacts were left on computers that were proof that the images were downloaded and viewed even if the images were deleted, just like a fingerprint on a murder weapon, it should be easy to convict each person.

The most anti-forensic material that was used by one of the culprits was heavy duty magnets that were installed in the shoes to erase the hard-drive of incriminating evidence. Yet with all the networking detective work, the magnets in the shoes probably just helped proof his guilt.

     Because the images were shared on a peer-to-peer network, every person involved in the arrest will not only be charged with possession of child pornography but also of distribution of child pornography because most torrent downloads automatically start uploading to other users who request the same data(Fletcher, 2012).

     30 year old Higinio O. Ochoa, a member of the hacker group Cabincr3w an offshoot of anonymous, was arrested after he posted an image of his girlfriend from an iPhone to Twitter. What he neglected to take into account was the GPS tagging EXIF metadata that was imprinted on the image. When the FBI viewed the metadata on the image, it effortlessly pointed to his girlfriend’s house in the outer-Melbourne area. Because of the image, I cannot post the actual image to this research paper but I can tell you that there was a message on it that his girlfriend was displaying, it read, “PwNd by W0rmer & cabinCr3w <3 u B(commented out)’s!”. All EXIF data had been wiped from the photos posted online.

     I was not able to find any current digital forensics tools that would look for coded messages, just encrypted messages. One helpful post I found from a digital forensics expert suggests that by using Unicode escape sequence messages, that you could possibly circumvent most digital forensic tools, unless it is a professional smart enough to check for the. For an example, \u0048 \u0045 \u004c \u004c \u004f , spells out HELLO.

     Fortunately there are people that are trying to close the gap for digital forensic tools lie Pavel Gladyshev of the UCD School of Computer Science and Informatics located in Texas, is working on a project to develop tools that will not only search for raw binary data for keywords but also search for possible character encoding to include ASCII, UTF-8, UTF-16, and UTF-32 that might have escape sequences embedded in it.

Anti-forensics for Your Protection

     Some people might jump to conclusions that by using anti-forensics to protect your information imply that you’re trying to hide illegal information. That is not always the case, sometimes it is useful to use anti-forensic tools in ordinary daily activities to protect against malware that targets devices like smartphones (Storm, 2011). Take for example the mobile forensic solutions offered by the company Cellebrite that are able to extract deleted data from all smartphones and tablets. While most information gleaned is produced from a hardwired connection, it is possible for devices to attach wireless through infrared or Bluetooth signal. The ability to access data remotely from a smart device makes forensic devices dangerous for the general populace because they may be used for criminal activity or spying (Bloomberg 2012).

      Companies like WhisperSystems (, make it a little bit harder for government and criminals alike to easily take data from your smart device by providing full disk encryption, network security tools, encrypted backup to the cloud, and selective permissions. Not only will anti-forensics software encrypt you data but it can also encrypt your text messages and voice calls if the other person is using the same software, if they are not it will still encrypt the data on your phone. This protection is not just necessary from a direct attack but also by malware that might disguise itself as an application you really want on you device.

    In the near future, I will be testing mobile digital forensic tools at Ferris State University and will test to see how well at least one of the free anti-forensic tools work during class and plan to come back and add more on forensics and security.


Berinato, S. (2007, June 8). The rise of anti-forensics. Retrieved from

Bloomberg Government, (March, 2012) IPhones to BlackBerrys Cracked by Cops Using Digital Forensics. Cellebrite mobile data secured. Retrieved 4/18/2012. From

Fletcher, J. (April, 2012). N.J. investigators track digital ‘fingerprints’ on shared images to nab child pornographers. The republic of Columbus Indiana. Retrieved 4/18/201, from