netcraft
/
litecoin
mirror of https://github.com/litecoin-project/litecoin
6
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge #19866: eBPF Linux tracepoints

Browse Source 
22eb7930a6 tracing: add tracing framework (William Casarin)
933ab8a720 build: detect sys/sdt.h for eBPF tracing (William Casarin)

Pull request description:

  Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin 😅 )

  The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages:

  # Examples

  ## Network Messages

  ```
  #!/usr/bin/env bpftrace

  BEGIN
  {
    printf("bitcoin net msgs\n");
    @start = nsecs;
  }

  usdt:./src/bitcoind:net:push_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @outbound[$command]++;
  }

  usdt:./src/bitcoind:net:process_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @inbound[$ip, $command]++;
  }

  ```

      $ sudo bpftrace netmsg.bt

  output: https://jb55.com/s/b11312484b601fb3.txt

  if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice!

  ## IBD Benchmarking

  ```
  #!/usr/bin/env bpftrace
  BEGIN
  {
    printf("IBD to 500,000 bench\n");
  }

  usdt:./src/bitcoind:CChainState:ConnectBlock
  {
    $height = (uint32)arg0;

    if ($height == 1) {
      printf("block 1 found, starting benchmark\n");
      @start = nsecs;
    }

    if ($height >= 500000) {
      @end = nsecs;
      @duration = @end - @start;
      exit();
    }
  }

  END {
    printf("duration %d ms\n", @duration / 1000000)
  }
  ```
  This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock

  Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging.

  Other ideas: tracepoints for lock contention, threads, what else?

  Let me know what ya'll think and if this is worth adding to bitcoin.

  ## TODO

  - [ ] docs?
  - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header)
  - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now

ACKs for top commit:
  laanwj:
    Tested ACK 22eb7930a6
  0xB10C:
    Tested ACK 22eb7930a6

Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
pull/826/head
Wladimir J. van der Laan 4 years ago
parent f7fd76bcc0 22eb7930a6
commit 43f3ada27b
No known key found for this signature in database
GPG Key ID: 1E4AED62986CD25D
3 changed files with 64 additions and 0 deletions
Show Stats Download Patch File Download Diff File

18
configure.ac
Unescape View File

@ -132,6 +132,12 @@ AC_ARG_WITH([bdb],
[use_bdb=$withval],
[use_bdb=auto])
AC_ARG_ENABLE([ebpf],
[AS_HELP_STRING([--enable-ebpf],
[enable eBPF tracing (default is yes if sys/sdt.h is found)])],
[use_ebpf=$enableval],
[use_ebpf=yes])
AC_ARG_WITH([miniupnpc],
[AS_HELP_STRING([--with-miniupnpc],
[enable UPNP (default is yes if libminiupnpc is found)])],
@ -1295,6 +1301,16 @@ if test x$enable_wallet != xno; then
fi
fi
if test x$use_ebpf != xno; then
AC_CHECK_HEADER([sys/sdt.h], [have_sdt=yes], [have_sdt=no])
else
have_sdt=no
fi
if test x$have_sdt = xyes; then
AC_DEFINE([ENABLE_TRACING], [1], [Define to 1 to enable eBPF user static defined tracepoints])
fi
dnl Check for libminiupnpc (optional)
if test x$use_upnp != xno; then
AC_CHECK_HEADERS(
@ -1689,6 +1705,7 @@ AM_CONDITIONAL([TARGET_WINDOWS], [test x$TARGET_OS = xwindows])
AM_CONDITIONAL([ENABLE_WALLET],[test x$enable_wallet = xyes])
AM_CONDITIONAL([USE_SQLITE], [test "x$use_sqlite" = "xyes"])
AM_CONDITIONAL([USE_BDB], [test "x$use_bdb" = "xyes"])
AM_CONDITIONAL([ENABLE_TRACING],[test x$have_sdt = xyes])
AM_CONDITIONAL([ENABLE_TESTS],[test x$BUILD_TEST = xyes])
AM_CONDITIONAL([ENABLE_FUZZ],[test x$enable_fuzz = xyes])
AM_CONDITIONAL([ENABLE_FUZZ_LINK_ALL],[test x$enable_danger_fuzz_link_all = xyes])
@ -1855,6 +1872,7 @@ echo " with bench = $use_bench"
echo " with upnp = $use_upnp"
echo " with natpmp = $use_natpmp"
echo " use asm = $use_asm"
echo " ebpf tracing = $have_sdt"
echo " sanitizers = $use_sanitizers"
echo " debug enabled = $enable_debug"
echo " gprof enabled = $enable_gprof"

1
src/Makefile.am
Unescape View File

@ -243,6 +243,7 @@ BITCOIN_CORE_H = \
util/system.h \
util/threadnames.h \
util/time.h \
util/trace.h \
util/translation.h \
util/ui_change_type.h \
util/url.h \

45
src/util/trace.h
Unescape View File

@ -0,0 +1,45 @@
// Copyright (c) 2020 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_UTIL_TRACE_H
#define BITCOIN_UTIL_TRACE_H
#ifdef ENABLE_TRACING
#include <sys/sdt.h>
#define TRACE(context, event) DTRACE_PROBE(context, event)
#define TRACE1(context, event, a) DTRACE_PROBE1(context, event, a)
#define TRACE2(context, event, a, b) DTRACE_PROBE2(context, event, a, b)
#define TRACE3(context, event, a, b, c) DTRACE_PROBE3(context, event, a, b, c)
#define TRACE4(context, event, a, b, c, d) DTRACE_PROBE4(context, event, a, b, c, d)
#define TRACE5(context, event, a, b, c, d, e) DTRACE_PROBE5(context, event, a, b, c, d, e)
#define TRACE6(context, event, a, b, c, d, e, f) DTRACE_PROBE6(context, event, a, b, c, d, e, f)
#define TRACE7(context, event, a, b, c, d, e, f, g) DTRACE_PROBE7(context, event, a, b, c, d, e, f, g)
#define TRACE8(context, event, a, b, c, d, e, f, g, h) DTRACE_PROBE8(context, event, a, b, c, d, e, f, g, h)
#define TRACE9(context, event, a, b, c, d, e, f, g, h, i) DTRACE_PROBE9(context, event, a, b, c, d, e, f, g, h, i)
#define TRACE10(context, event, a, b, c, d, e, f, g, h, i, j) DTRACE_PROBE10(context, event, a, b, c, d, e, f, g, h, i, j)
#define TRACE11(context, event, a, b, c, d, e, f, g, h, i, j, k) DTRACE_PROBE11(context, event, a, b, c, d, e, f, g, h, i, j, k)
#define TRACE12(context, event, a, b, c, d, e, f, g, h, i, j, k, l) DTRACE_PROBE12(context, event, a, b, c, d, e, f, g, h, i, j, k, l)
#else
#define TRACE(context, event)
#define TRACE1(context, event, a)
#define TRACE2(context, event, a, b)
#define TRACE3(context, event, a, b, c)
#define TRACE4(context, event, a, b, c, d)
#define TRACE5(context, event, a, b, c, d, e)
#define TRACE6(context, event, a, b, c, d, e, f)
#define TRACE7(context, event, a, b, c, d, e, f, g)
#define TRACE8(context, event, a, b, c, d, e, f, g, h)
#define TRACE9(context, event, a, b, c, d, e, f, g, h, i)
#define TRACE10(context, event, a, b, c, d, e, f, g, h, i, j)
#define TRACE11(context, event, a, b, c, d, e, f, g, h, i, j, k)
#define TRACE12(context, event, a, b, c, d, e, f, g, h, i, j, k, l)
#endif
#endif /* BITCOIN_UTIL_TRACE_H */
Write Preview
Loading…
Cancel
Save