Skip to main content
If you like Apache Cloudberry (Incubating), give it a star on GitHub!

pg_filedump

Filedump, also known as the command-line tool pg_filedump, is a system utility that formats Cloudberry Database heap files, index files, and control files into human-readable forms. Using filedump, you can format and dump files in multiple ways, as described in the Command-line options section, and even dump binary data directly.

The repository for Cloudberry Database's filedump can be found at: https://github.com/cloudberrydb/filedump

User scenarios​

The filedump tool is primarily used in the following scenarios:

  • File analysis and debugging: Converts heap, index, or control files from the database into a readable format, helping file content analysis and database issue debugging.
  • Data recovery: Helps extract and recover data when database files are corrupted.
  • Database learning and research: Provides developers and administrators a way to directly inspect database file contents, which is useful for studying database internals or learning about file formats.

Compile and install filedump​

Before compiling filedump, you need to have a Cloudberry Database cluster installed with version 1.0.0 or later. The following are the typical steps for compilation:

  1. Ensure the CloudberryDB package can be found:

    su - gpadmin
    source /usr/local/cloudberry-db/greenplum_path.sh

  2. Clone the GitHub repository cloudberrydb/filedump to your local environment:

    git clone https://github.com/cloudberrydb/filedump.git

  3. Enter the repository directory and run the compilation command:

    cd filedump
    make

  4. Install pg_filedump:

    make install

Configuration options​

The following configuration options are available during the compilation process:

  • ENABLE_ZLIB: Enables the use of pg_filedump on zlib-compressed data files. If the zlib headers and libraries are detected, zlib support will be automatically enabled. You can force zlib support to be enabled with the following command:

    make -f Makefile ENABLE_ZLIB=y

    You can also force it to be disabled by setting this variable to n.

  • ENABLE_ZSTD: Similar settings are used for the zstd compression method.

Usage examples​

Assuming you have successfully installed filedump, here are some common command examples:

Format a heap file​

pg_filedump -f /path/to/heap/file

Example:

$ pg_filedump -f /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385

*******************************************************************
* PostgreSQL File/Block Formatted Dump Utility
*
* File: /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385
* Options used: -f
*******************************************************************

Block    0 ********************************************************
<Header> -----
Block Offset: 0x00000000         Offsets: Lower      44 (0x002c)
Block: Size 32768  Version   14            Upper    32568 (0x7f38)
LSN:  logid      0 recoff 0x0c084ef8      Special  32768 (0x8000)
Items:    5                      Free Space: 32524
Checksum: 0x27ac  Prune XID: 0x00000000  Flags: 0x0000 ()
Length (including item array): 44

0000: 00000000 f84e080c ac270000 2c00387f  .....N...'..,.8.
0010: 00800e80 00000000 d8ff4800 b0ff5000  ..........H...P.
0020: 88ff5000 60ff5000 38ff5000           ..P.`.P.8.P.    

<Data> -----
Item   1 -- Length:   36  Offset: 32728 (0x7fd8)  Flags: NORMAL
7fd8: 1b030000 00000000 00000000 00000000  ................
7fe8: 01000300 02081800 02000000 09426f62  .............Bob
7ff8: 19000000                             ....            

Item   2 -- Length:   40  Offset: 32688 (0x7fb0)  Flags: NORMAL
7fb0: 1b030000 00000000 00000000 00000000  ................
7fc0: 02000300 02081800 03000000 11436861  .............Cha
7fd0: 726c6965 23000000                    rlie#...        

Item   3 -- Length:   40  Offset: 32648 (0x7f88)  Flags: NORMAL
7f88: 1c030000 00000000 00000000 00000000  ................
7f98: 03000300 02081800 04000000 0d446176  .............Dav
7fa8: 69640000 1c000000                    id......        

Item   4 -- Length:   40  Offset: 32608 (0x7f60)  Flags: NORMAL
7f60: 1d030000 00000000 00000000 00000000  ................
7f70: 04000300 02081800 07000000 11436861  .............Cha
7f80: 726c6965 23000000                    rlie#...        

Item   5 -- Length:   40  Offset: 32568 (0x7f38)  Flags: NORMAL
7f38: 1d030000 00000000 00000000 00000000  ................
7f48: 05000300 02081800 08000000 0d446176  .............Dav
7f58: 69640000 1c000000                    id......        

*** End of File Encountered. Last Block Read: 0 ***

View the content of a control file​

pg_filedump -c /path/to/pg_control

Example:

$ pg_filedump -c /home/gpadmin/datadirs/qddir/demoDataDir-1/global/pg_control

*******************************************************************
* PostgreSQL File/Block Formatted Dump Utility
*
* File: /home/gpadmin/datadirs/qddir/demoDataDir-1/global/pg_control
* Options used: -c
*******************************************************************

<pg_control Contents> *********************************************

                        CRC: Correct
        pg_control Version: 13000700
            Catalog Version: 302406171
            System Identifier: 7405502447288557259
                        State: IN PRODUCTION
                Last Mod Time: Wed Aug 21 16:20:06 2024
    Last Checkpoint Record: Log File (0) Offset (0x0c088f80)
Last Checkpoint Record Redo: Log File (0) Offset (0x0c088f48)
        |-       TimeLineID: 1
        |-         Next XID: 0/809
        |-         Next OID: 24576
        |- Next Relfilenode: 24576
        |-       Next Multi: 1
        |-    Next MultiOff: 0
        |-             Time: Wed Aug 21 16:20:06 2024
    Minimum Recovery Point: Log File (0) Offset (0x00000000)
        Backup Start Record: Log File (0) Offset (0x00000000)
            Backup End Record: Log File (0) Offset (0x00000000)
End-of-Backup Record Required: no
    Maximum Data Alignment: 8
        Floating-Point Sample: 1234567
        Database Block Size: 32768
        Blocks Per Segment: 32768
            XLOG Block Size: 32768
            XLOG Segment Size: 67108864
    Maximum Identifier Length: 64
        Maximum Index Keys: 32
            TOAST Chunk Size: 8140

Format and display a specific block range​

pg_filedump -R 0 10 -f /path/to/heap/file

Example:

$ pg_filedump -R 0 10 -f /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385

*******************************************************************
* PostgreSQL File/Block Formatted Dump Utility
*
* File: /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385
* Options used: -R 0 10 -f
*******************************************************************

Block    0 ********************************************************
<Header> -----
Block Offset: 0x00000000         Offsets: Lower      44 (0x002c)
Block: Size 32768  Version   14            Upper    32568 (0x7f38)
LSN:  logid      0 recoff 0x0c084ef8      Special  32768 (0x8000)
Items:    5                      Free Space: 32524
Checksum: 0x27ac  Prune XID: 0x00000000  Flags: 0x0000 ()
Length (including item array): 44

0000: 00000000 f84e080c ac270000 2c00387f  .....N...'..,.8.
0010: 00800e80 00000000 d8ff4800 b0ff5000  ..........H...P.
0020: 88ff5000 60ff5000 38ff5000           ..P.`.P.8.P.    

<Data> -----
Item   1 -- Length:   36  Offset: 32728 (0x7fd8)  Flags: NORMAL
7fd8: 1b030000 00000000 00000000 00000000  ................
7fe8: 01000300 02081800 02000000 09426f62  .............Bob
7ff8: 19000000                             ....            

Item   2 -- Length:   40  Offset: 32688 (0x7fb0)  Flags: NORMAL
7fb0: 1b030000 00000000 00000000 00000000  ................
7fc0: 02000300 02081800 03000000 11436861  .............Cha
7fd0: 726c6965 23000000                    rlie#...        

Item   3 -- Length:   40  Offset: 32648 (0x7f88)  Flags: NORMAL
7f88: 1c030000 00000000 00000000 00000000  ................
7f98: 03000300 02081800 04000000 0d446176  .............Dav
7fa8: 69640000 1c000000                    id......        

Item   4 -- Length:   40  Offset: 32608 (0x7f60)  Flags: NORMAL
7f60: 1d030000 00000000 00000000 00000000  ................
7f70: 04000300 02081800 07000000 11436861  .............Cha
7f80: 726c6965 23000000                    rlie#...        

Item   5 -- Length:   40  Offset: 32568 (0x7f38)  Flags: NORMAL
7f38: 1d030000 00000000 00000000 00000000  ................
7f48: 05000300 02081800 08000000 0d446176  .............Dav
7f58: 69640000 1c000000                    id......        



*** End of File Encountered. Last Block Read: 0 ***

Get detailed information using the -i and -f options​

pg_filedump -i -f /path/to/heap/file

Example:

$ pg_filedump -i -f /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385

*******************************************************************
* PostgreSQL File/Block Formatted Dump Utility
*
* File: /home/gpadmin/datadirs/dbfast1/demoDataDir0/base/13347/16385
* Options used: -i -f
*******************************************************************

Block    0 ********************************************************
<Header> -----
 Block Offset: 0x00000000         Offsets: Lower      44 (0x002c)
 Block: Size 32768  Version   14            Upper    32568 (0x7f38)
 LSN:  logid      0 recoff 0x0c084ef8      Special  32768 (0x8000)
 Items:    5                      Free Space: 32524
 Checksum: 0x27ac  Prune XID: 0x00000000  Flags: 0x0000 ()
 Length (including item array): 44

  0000: 00000000 f84e080c ac270000 2c00387f  .....N...'..,.8.
  0010: 00800e80 00000000 d8ff4800 b0ff5000  ..........H...P.
  0020: 88ff5000 60ff5000 38ff5000           ..P.`.P.8.P.    

<Data> -----
 Item   1 -- Length:   36  Offset: 32728 (0x7fd8)  Flags: NORMAL
  XMIN: 795  XMAX: 0  CID|XVAC: 0
  Block Id: 0  linp Index: 1   Attributes: 3   Size: 24
  infomask: 0x0802 (HASVARWIDTH|XMAX_INVALID) 

  7fd8: 1b030000 00000000 00000000 00000000  ................
  7fe8: 01000300 02081800 02000000 09426f62  .............Bob
  7ff8: 19000000                             ....            

 Item   2 -- Length:   40  Offset: 32688 (0x7fb0)  Flags: NORMAL
  XMIN: 795  XMAX: 0  CID|XVAC: 0
  Block Id: 0  linp Index: 2   Attributes: 3   Size: 24
  infomask: 0x0802 (HASVARWIDTH|XMAX_INVALID) 

  7fb0: 1b030000 00000000 00000000 00000000  ................
  7fc0: 02000300 02081800 03000000 11436861  .............Cha
  7fd0: 726c6965 23000000                    rlie#...        

 Item   3 -- Length:   40  Offset: 32648 (0x7f88)  Flags: NORMAL
  XMIN: 796  XMAX: 0  CID|XVAC: 0
  Block Id: 0  linp Index: 3   Attributes: 3   Size: 24
  infomask: 0x0802 (HASVARWIDTH|XMAX_INVALID) 

  7f88: 1c030000 00000000 00000000 00000000  ................
  7f98: 03000300 02081800 04000000 0d446176  .............Dav
  7fa8: 69640000 1c000000                    id......        

 Item   4 -- Length:   40  Offset: 32608 (0x7f60)  Flags: NORMAL
  XMIN: 797  XMAX: 0  CID|XVAC: 0
  Block Id: 0  linp Index: 4   Attributes: 3   Size: 24
  infomask: 0x0802 (HASVARWIDTH|XMAX_INVALID) 

  7f60: 1d030000 00000000 00000000 00000000  ................
  7f70: 04000300 02081800 07000000 11436861  .............Cha
  7f80: 726c6965 23000000                    rlie#...        

 Item   5 -- Length:   40  Offset: 32568 (0x7f38)  Flags: NORMAL
  XMIN: 797  XMAX: 0  CID|XVAC: 0
  Block Id: 0  linp Index: 5   Attributes: 3   Size: 24
  infomask: 0x0802 (HASVARWIDTH|XMAX_INVALID) 

  7f38: 1d030000 00000000 00000000 00000000  ................
  7f48: 05000300 02081800 08000000 0d446176  .............Dav
  7f58: 69640000 1c000000                    id......        



*** End of File Encountered. Last Block Read: 0 ***

Command-line options​

When using the filedump tool, you can format and dump database files with the following command:

pg_filedump [-abcdfhikxy] [-R startblock [endblock]] [-D attrlist] [-S blocksize] [-s segsize] [-n segnumber] file

The default settings are: relative addressing, formatting the entire file, and using the block size listed in block 0 of the file.

The following options are available for heap and index files:

  • -a: Displays absolute addresses when formatting. Block header information is always block-relative.
  • -b: Displays binary block images within a specified range. This option will disable all other formatting options.
  • -d: Displays a formatted block content dump. This option will disable all other formatting options.
  • -D: Decodes tuples using a given comma-separated list of types. Supported types include:
    • bigint
    • bigserial
    • bool
    • char
    • charN -- char(n)
    • date
    • float
    • float4
    • float8
    • int
    • json
    • macaddr
    • name
    • oid
    • real
    • serial
    • smallint
    • smallserial
    • text
    • time
    • timestamp
    • timetz
    • uuid
    • varchar
    • varcharN -- varchar(n)
    • xid
    • xml
    • ~: Ignores all remaining attributes in the tuple.
  • -f: Displays a formatted block content dump along with interpretation.
  • -h: Displays help information.
  • -i: Displays interpreted item details.
  • -k: Verifies block checksums.
  • -R: Displays specific block ranges within the file (Blocks are indexed from 0).
    • [startblock]: The block to start at.
    • [endblock]: The block to end at (exclusive). If only startblock is specified, only that single block will be formatted.
  • -s: Forces the segment size to [segsize].
  • -n: Forces the segment number to [segnumber].
  • -S: Forces the block size to [blocksize].
  • -x: Forces interpreted formatting of block items as index items.
  • -y: Forces interpreted formatting of block items as heap items.

The following options are available for control files:

  • -c: Interprets the specified file as a control file.
  • -f: Displays a formatted content dump along with interpretation.
  • -S: Forces the block size to [blocksize].

It is recommended to use the -i and -f options to obtain the most useful dump output.

Support
GitHub Issues
GitHub Discussions
Slack
Twitter
Youtube
Security
Resources
Download
Documentation
Events
Code of Conduct
Brand Guidelines
Contribution
Working with Git & GitHub
Contribution Overview
Code Contribution
Doc Contribution

Apache Cloudberry is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Copyright © 2024 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.

Apache®, the names of Apache projects, and the feather logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.