Date :
2011.10.18
Category :
OS - Linux - Analysis - Tool
Note :
Ubuntu에서의 설치는 Document #3 을 참조.
Centos에서의 설처는 Document #4 를 참조.
#1
Source :
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-Realtime_Specific_Tuning-Using_the_ftrace_Utility_for_Tracing_Latencies.html
Content :
3.7. Using the ftrace
Utility for Tracing Latencies
ftrace
, which is used to determine where a desired maximum latency is being exceeded. The ftrace
utility has a variety of options that allow you to use the utility in a number of different ways. The utility is particularly useful for identifying whether non-deterministic performance results are attributable to the kernel or to user space components. In customer deployments, the tool is most useful to differentiate whether delays are in the kernel or the application. ftrace
utility is not enabled in the production version of the MRG Realtime kernel as it creates additional overhead. If you wish to use the ftrace
utility you will need to download and install the trace
variant of the MRG Realtime kernel from the mrg-beta
yum repository. Note
mrg-beta
yum repository, see the MRG Realtime Installation Guide. Using the ftrace
Utility
-
Once you are using the
trace
variant of the MRG Realtime kernel, you can set up theftrace
utility. You will need to create a/debug
directory and then mount it to use thedebugfs
file system.# mkdir /debug # mount -t debugfs nodev /debug
-
To check if the
ftrace
utility is running, use thecat
command to view the/debug/tracing/tracing_enabled
file. A value of1
indicates thatftrace
is running, and0
indicates that it is not running.# cat /debug/tracing/tracing_enabled 1
By default, the tracer is enabled. To turn the tracer on or off,echo
the appropriate value to the/debug/tracing/tracing_enabled
file.# echo 0 > /debug/tracing/tracing_enabled # echo 1 > /debug/tracing/tracing_enabled
-
The
ftrace
utility has a variety of options that allow you to use the utility in a number of different ways. The available options are:events
Trace specific events. wakeup
Record the time it takes for the highest priority RT task to wake up. If no RT tasks are running, this option will result in no output. pre-emptoff
Record the the longest time for which pre-emption is disabled. irqsoff
Record the the longest time for which interrupts have been disabled. pre-emptirqsoff
Record the the longest time for which either interrupts or pre-emption have been disabled. ftrace
Trace functions using mcount
.sched_switch
Trace tasks as they are switched by the scheduler. none
Disable all tracing. -
To check which options you currently have in the tracer use the
cat
command on the/debug/tracing/available_tracers
file:# cat /debug/tracing/available_tracers
-
To set a single option on the tracer,
echo
the option name to the/debug/tracing/current_tracer
file.# echo ftrace > /debug/tracing/current_tracer
Note
If you use a single>
with theecho
command, it will override any existing value in the file. If you wish to append the value to the file, use>>
instead.
-
The results of any traces are saved to two files. The contents of
/debug/tracing/latency_trace
contains detailed information:# cat /debug/tracing/latency_trace
The/debug/tracing/trace
uses a simpler format:# cat /debug/tracing/trace
-
There are a number of options available for changing the format of the output. These options are stored in
/debug/tracing/iter_ctrl
:print-parent
Show the parent of the functions. sym-offset
Add the offset into the function. sym-addr
Add the address of a symbol. verbose
Increase the verbosity of the tracer output.
-
Use the
cat
command to view the current configuration:# cat /debug/tracing/iter_ctrl
To set a single option on the tracer output configuration,echo
the option name to the/debug/tracing/iter_ctrl
file.# echo verbose > /debug/tracing/iter_ctrl
To disable a single option on the tracing output configuration,echo
the option name with the testno
before it to the/debug/tracing/iter_ctrl
file.# echo noverbose > /debug/tracing/iter_ctrl
Note
If you use a single>
with theecho
command, it will override any existing value in the file. If you wish to append the value to the file, use>>
instead.
-
The
ftrace
utility can be filtered by altering the settings in the/debug/tracing/set_ftrace_filter
file. If no filters are specified in the file, all processes are traced. Use thecat
to view the current filters:# cat /debug/tracing/set_ftrace_filter
-
To change the filters,
echo
the name of the process to be traced. The filter allows the use of a*
wildcard at the beginning or end of a search term. Some examples of filters are:-
Trace only the
schedule
process:# echo schedule > /debug/tracing/set_ftrace_filter
-
Trace all processes that end with
lock
:# echo *lock > /debug/tracing/set_ftrace_filter
-
Trace all processes that start with
spin_
:# echo spin_* > /debug/tracing/set_ftrace_filter
-
Trace all processes with
cpu
in the name:# echo *cpu* > /debug/tracing/set_ftrace_filter
Note
The*
wildcard for the tracer filter will only work at the beginning or end of a word. For example:spin_*
and*lock
will work, butspin_*lock
will not.
- Document :
#2
Source :
http://linux.die.net/man/1/ftrace
Content :
ftrace(1) - Linux man page
Name
ftrace - Runs the frysk systemcall tracing utility
Synopsis
ftrace
Description
ftrace Runs the frysk systemcall tracing utility.
ftrace is a small utility that uses the frysk engine to trace systemcalls in a similar manner to strace.
Example
ftrace command
ftrace pid
See Also
funit(8) fcatch(1) fcore(1) fstack(1) fstep(1)
Bugs
The option parser is greedy when looking for options so running ftrace on a program that uses options can be a problem, use -- to split between ftrace and the program. So change from:
~/prefix/bin/ftrace ~/prefix/lib64/frysk/funit --arch 32 frysk.proc.TestAbandon
to
~/prefix/bin/ftrace -- ~/prefix/lib64/frysk/funit --arch 32 frysk.proc.TestAbandon
Report bugs to [1] http://sources.redhat.com/ftrace/ .
References
1. http://sources.redhat.com/ftrace/
http:/sources.redhat.com/ftrace
Referenced By
fhpd(1), fltrace(1), frysk(1), frysk(7)
#3
Source :
http://www.mgriepentrog.com/2009/09/09/enable-ftrace-on-ubuntu-linux/
Content :
Enable ftrace on Ubuntu Linux Introduction
I was interested in using ftrace for a potential project, but didn’t find much documentation on how to get it working. I hope to change that by providing step-by-step instructions to enable ftrace for use in Ubuntu Linux 9.10 (Karmic Koala). Overview from the kernel documentation:
Ftrace is an internal tracer designed to help out developers and designers of systems to find what is going on inside the kernel. It can be used for debugging or analyzing latencies and performance issues that take place outside of user-space.
Given that ftrace is targeted towards developers and designers of systems, I’m assuming you more or less know what you’re doing with these steps. I also recommend virtualizing an instance of linux so that if you do mess up, you aren’t left with an un-bootable machine.
Steps
Download and install Ubuntu 9.10 Alpha-3
Run the following commands to install the necessary dependencies for a custom kernel:
$ sudo apt-get install git-core fakeroot build-essential makedumpfile kernel-wedge
$ sudo apt-get build-dep linuxGet the latest version of the Ubuntu kernel:
$ cd ~/
$ git clone git://kernel.ubuntu.com/ubuntu/ubuntu-karmic.gitedit “debian/config/config.common.ubuntu” and make the following changes:
CONFIG_FTRACE_SYSCALLS=y
CONFIG_FUNCTION_TRACER=yNext, run the following commands:
$ sudo debian/rules clean
$ CONCURRENCY_LEVEL=2 AUTOBUILD=1 NOEXTRAS=1 fakeroot debian/rules binary-386If you are prompted about enabling other forms of tracing, see ftrace.txt for more information about each type. You will want to enable DYNAMIC_FTRACE, which allows you to only trace functions you are concerned about.
Once finished, run the following commands from the parent directory :
$ sudo dpkg -i linux-image-2.6.31-5-386_2.6.31-5.24_i386.deb
$ sudo dpkg -i linux-headers-2.6.31-5-386_2.6.31-5.24_i386.deb
$ sudo update-grubRestart your computer, and you should now see the kernel we compiled as a boot option:
The rest is outlined in LWN’s look at ftrace, but here are a few things I noticed:
- You don’t have to mount a debugfs. There’s already one at /sys/kernel/debug (where it’s recommended to be).
- I ran into some permission errors when trying to change what the current tracer was. Changing the owner or permissions of that directory should solve this issue.
To verify that everything is working, when you cat “/sys/kernel/debug/tracing/available_tracers” you should see more options than just “nop”. Enjoy!
Resources
- Ubuntu Kernel Compile Page
- How to compile a custom kernel for Ubuntu Intrepid using git
- ftrace.txt
- LWN.net: A look at ftrace
http://serverfault.com/questions/268037/centos-5-realtime-patch
2 Have you gone searching through the CentOS forums? A quick Google search yeilded a few hits. Also why do you need the real time patch? Perhaps changing your process scheduler to something like the deadline scheduler might be just as suitable. – Red Tux May 10 at 7:18@Red Tux: I tried to search and I got only MRG project, but I prefer to use something free. About deadline scheduler: I've never heard of it. I don't know what better for me, I'm implementing server that receiving big number of UDP packets per seconds and I got missing packets even I set receiver thread priority to 99 with SCHED_FIFO. I think, to use real-time path will be enough? – Dima May 10 at 18:06This is right in the sweet-spot for a realtime kernel. You will notice a difference. The benefit of the solution I posted below is that you will be able to test the realtime kernel without making any major changes to your operating system installation. RHEL MRG is an add-on to the base Enterprise Linux product. – ewwhite May 10 at 18:14Was this post useful to you?While I own a few RHEL MRG licenses, I've started using the FREE realtime recompile of Red Hat MRG from Scientific Linux to convert my CentOS 5 servers to realtime. You can view the Scientific Linux MRG distribution at: http://glitesoft.cern.ch/cern/mrg/slc5X/x86_64/RPMS/
In order to install, I create a small repo file named
slc5x-mrg-realtime.repo
and drop it in/etc/yum.repos.d
. The file contains:[slc-mrg-realtime] name = SLC5X - MRG Realtime baseurl = http://linuxsoft.cern.ch/cern/mrg/slc5X/$basearch/RPMS enabled = 1 protect = 0 gpgkey = http://glitesoft.cern.ch/cern/slc44/x86_64/RPM-GPG-KEYs/RPM-GPG-KEY-cern gpgcheck = 1
From there, I can simply run
yum grouplist
and see the MRG component groups:MRG Realtime MRG Grid MRG Management MRG Messaging
A
yum groupinstall "MRG Realtime" && yum install kernel-rt-devel
gives me everything I need for realtime operation. You can also just pick and choose the files you want from the distro's rpm listing page at: http://glitesoft.cern.ch/cern/mrg/slc5X/x86_64/RPMS/
Another option is to perform a recompile of Red Hat's SRPMS available at: http://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/RHEMRG/SRPMS/
It's the standard
rpmbuild --rebuild package.srpm
process. This takes more time, but I've done this successfully as well.I prefer to use free SW (not commercial MRG) How can I know, which RT path I need for CentOS 5.5 kernel? My Kernel version is: 2.6.18-194.26.1.el5, as I see, the latest version of the RT patch for 2.6.18 is patch-2.6.18-rt7. Is it OK to use use it? – Dima May 15 at 13:191 Please reread my comment. I've provided you two free solutions. Scientific Linux IS free! It has a clone of RHEL MRG in the sane manner that CentOS is a clone of RHEL. MRG is a new kernel and a set of utilities, not a patch. – ewwhite May 15 at 14:32
-
'Analysis' 카테고리의 다른 글
dstat (0) | 2011.11.25 |
---|---|
Intel VTune (0) | 2011.11.23 |
HP Glance Plus (0) | 2011.11.15 |
oprofile (1) | 2011.11.11 |
strace, ltrace, ftrace (0) | 2011.11.07 |