It might be related to the kernel’s watchdog code due to research by Linus Torvalds. “So I’m looking at the watchdog code, and it seems racy [with regard to] parking and startup…Quite frankly, I’m just grasping for straws here, but a lot of the watchdog traces really have seemed spurious…”
via [Phoronix] Nasty Lockup Issue Still Being Investigated For Linux 3.18.
Unlike a normal virus scanner on consumer PCs that compares a catalog of known viruses to something that has infected the computer, A3 can detect new, unknown viruses or malware automatically by sensing that something is occurring in the computer’s operation that is not correct. It then can stop the virus, approximate a repair for the damaged software code, and then learn to never let that bug enter the machine again.
via Self-repairing software tackles malware — ScienceDaily.
The A3 software is open source, meaning it is free for anyone to use, but Eide believes many of the A3 technologies could be incorporated into commercial products
Download papers from the source: A3 : Flux Research Group
The A3 project applies virtualization, record-and-replay, introspection, repair, and other techniques to develop a customizable container for “advanced adaptive applications.” The A3 container provides its protected application with both innate and adaptive defenses against security threats.
Available Software
Stackdb — a VMI-enabled debugging library for multi-level systems (read the paper; browse the source code; git clone the source repository)
Weir — a streaming language for systems analysis (read the paper; browse the source code; git clone the source repository)
XenTT — a “time-traveling” hypervisor (docs and source code)
#!/bin/bash
umask 077
max_lines=10000
linecount=$(wc -l < ~/.bash_history)
if (($linecount > $max_lines)); then
prune_lines=$(($linecount - $max_lines))
head -$prune_lines ~/.bash_history >> ~/.bash_history.archive \
&& sed -e "1,${prune_lines}d" ~/.bash_history > ~/.bash_history.tmp$$ \
&& mv ~/.bash_history.tmp$$ ~/.bash_history
fi
via BashFAQ/088 – Greg’s Wiki.
I needed to manage shell command history in a formal fashion in order to turn repeated sequences into scripts without having to type them in again. I also wanted a record of packages installed and in what order. The history of commands is contained in .bash_history file which is read once when a terminal opens. Running set -o vi allows for history commands to be recalled using standard vi commands. The above script can be run as a user level cron job to periodically prune the top so many commands and place them into an archive.
The bash statements below set history size and make it so a command will be written to the history file immediately and not simply when a terminal closes. These should be placed in .bashrc or whatever file executes when a new terminal opens.
HISTFILESIZE=400000000 HISTSIZE=10000 PROMPT_COMMAND="history -a" export HISTSIZE PROMPT_COMMAND shopt -s histappend
Usually whenever we have to remove duplicate entries from a file, we do a sort of the entries and then eliminate the duplicates using “uniq” command.
But if we have to remove the duplicates and preserve the same order of occurrence of the entries, here is the way:
via UNIX Command Line: Remove duplicates without sorting file – BASH.
$ awk ‘ !x[$0]++’ file3
From: Unix: removing duplicate lines without sorting
This command is simply telling awk which lines to print. The variable $0 holds the entire contents of a line and square brackets are array access. So, for each line of the file, the node of the array named x is incremented and the line printed if the content of that node was not (!) previously set.
[rmiller@pacific]# echo "hello world" > hello
[rmiller@pacific]# xxd hello
0000000: 6865 6c6c 6f20 776f 726c 640a hello worldSo you can use this tool to byte edit files. One rather unusual use I’ve found for it is to paste in an RPM to a system that I only had serial console access to. I just ran xxd on it, copied it into the buffer, and pasted it into a file on the remote server. A quick xxd -r, and voila. RPM.
I recently ran across the above blog entry which is from 2010. All these years working with Unix systems and I never knew about this command. When I parse a web site to extract information it’s necessary to output clean and concise ASCII data for my downstream scripts. My perl scripts that html parse do filter this out but sometimes a funny character gets through. Normally I have been using hexedit to determine the hex code of the offending character and although it works, it’s not as elegant as the above xxd command. Now I can do the following:
xxd offendingdatafile.txt | grep "mystring" | more
The above should simply output lines containing the offending hexcode using grep if I kind of notice a unique searchable string (mystring) before the offending hex character. I could also:
xxd offendingdatafile.txt > myfile.dat
vi myfile.dat
Instead of using clunky hexedit to search for mystring I can use good old vi.
I’m sure there are lots of other uses for this utility — especially in shell scripts. Unix has so many commands and I utilize a subset adequate to getting whatever it is I need to do. Every year I pick up one or two new useful commands that are more efficient and xxd is one of them.
env x='() { :;}; echo OOPS' bash -c :
The “env” command runs a command with a given variable set. In this case, we’re setting “x” to something that looks like a function. The function is just a single “:”, which is actually a simple command which is defined as doing nothing. But then, after the semi-colon which signals the end of the function definition, there’s an echo command. That’s not supposed to be there, but there’s nothing stopping us from doing it.
via Shellshock: How does it actually work? | Fedora Magazine.
But — oops! When that new shell starts up and reads the environment, it gets to the “x” variable, and since it looks like a function, it evaluates it. The function definition is harmlessly loaded — and then our malicious payload is triggered too. So, if you run the above on a vulnerable system, you’ll get “OOPS” printed back at you. Or, an attacker could do a lot worse than just print things.
I copied and pasted the above env command and it echos back OOPS. This web server has been (I suspect) scanned already once with the scanner placing a ping command in the User Agent HTTP field. Apparently User Agent gets passed to a shell environmental variable which will then get executed. The only problem is that they need some kind of script to execute which there are none on this site. This site simply returned 404, file not found to the scanner.
This could be problematic on sites with a lot of cgi scripts. There is some exploit that can affect a client using dhcp to obtain an IP address from a malicious server. I’ll find an explanation of that and put that up in its own post. This story is evolving and even has its own brand name now — shellshock.
Developed in large part by now defunct Willow Garage, ROS was designed for collaborative, open source robotics development. ROS is a collection of tools and libraries that simplify the task of creating and programming robotic platforms and applications. ROS is not a real-time OS, but it can be integrated with RTOSes, as well as Linux. As of earlier this year, Android support was added.
The default install for ROS is Ubuntu Linux. Core ROS components include message passing, message recording and playback, remote procedure calls, and a distributed parameter system. In addition to these core middleware components, ROS offers more robotics-specific features like a Unified Robot Description Format (URDF), a remote geometry library, preemptable remote procedure calls, and diagnostics. It also offers ready-built packages for common robotics problems like mobile navigation, pose estimation, and building a map and having the robot self-localize on it.
via Robot OS to support Linux and Android on Snapdragon · LinuxGizmos.com.
The main tenets of the Unix Philosophy are as follows::
- Small is beautiful.
- Make each program do one thing well.
- Build a prototype as soon as possible.
- Choose portability over efficiency.
- Store data in flat text files.
- Use software leverage to your advantage.
- Use shell scripts to increase leverage and portability.
- Avoid captive user interfaces.
- Make every program a filter.
In this tutorial, we’ll develop a simple kernel module that creates a /dev/reverse device. A string written to this device is read back with the word order reversed (“Hello World” becomes “World Hello”). It is a popular programmer interview puzzle, and you are likely to get some bonus points when you show the ability to implement it at the kernel level as well. A word of warning before we start: a bug in your module may lead to a system crash and (unlikely, but possible) data loss.
Enter GNU Parallel, a shell tool designed for executing tasks in parallel using one or more computers. For my purposes I just ran in on a single system, but wanted to take advantage of multiple cores.
Having enough memory on my system, I loaded the entire massive file into memory and pipe it to GNU Parallel along with another file consisting of thousands of different strings I want to search for in the “PATTERNFILE”:
cat BIGFILE | parallel –pipe grep -f PATTERNFILE
via Speeding Up Grep Log Queries with GNU Parallel – The State of Security.