nodes deleted Vs. nodes physically delete from hard drive (Reclaim unused space)Mar 20, 2024Knowledge
Article DetailsDetailsHelp DetailsWhat will happen when we delete a node or a relationship in Neo4j database? After you get a response message from Neo4j database that tell you the node or relationship has been successfully delete, in reality, at the backend, it just means that the storage is marked as "deleted" and will be used later when new nodes or relationship is created. It also means after the deletion, Neo4j database is still managing the storage space, the total storage space of the database will not shrink at this point. If you want to compress the db to remove those records and optimise the space used, they need to use neo4j-admin copy (or, if on 3.5.x or below, the store-utils). |07:55:40|Neo4j:[import]> neo4j-admin copy --to-database=testdumpcopy1 --from-database=testdump --force --verbose neo4j 4.2.4 VM Name: Java HotSpot(TM) 64-Bit Server VM VM Vendor: Oracle Corporation VM Version: 11.0.9+7-LTS JIT compiler: HotSpot 64-Bit Tiered Compilers VM Arguments: [-XX:+UseParallelGC, -Xms4096m, -Xmx8192m, -Dfile.encoding=UTF-8] Starting to copy store, output will be saved to: /Users/fyao/softwares/neo4j-enterprise-4.2.4/logs/neo4j-admin-copy-2021-06-08.09.56.48.log 2021-06-08 16:56:49.573+0000 INFO [StoreCopy] ### Copy Data ### 2021-06-08 16:56:49.573+0000 INFO [StoreCopy] Source: /Users/fyao/softwares/neo4j-enterprise-4.2.4/data/databases/testdump (page cache 8m) 2021-06-08 16:56:49.573+0000 INFO [StoreCopy] Target: /Users/fyao/softwares/neo4j-enterprise-4.2.4/data/databases/testdumpcopy1 (page cache 8m) 2021-06-08 16:56:49.574+0000 INFO [StoreCopy] Empty database created, will start importing readable data from the source. 2021-06-08 16:56:53.700+0000 INFO [o.n.i.b.ImportLogic] Import starting Nodes, started 2021-06-08 16:56:56.488+0000 [*Nodes:0B/s 30.86MiB-------------------------------------------------------------------------]6.15M ∆ 450K Done in 11s 137ms Prepare node index, started 2021-06-08 16:57:07.781+0000 [*DETECT:39.12MiB-----------------------------------------------------------------------------]7.07M ∆60.0K Done in 766ms Relationships, started 2021-06-08 16:57:08.554+0000 [*Relationships:?? 39.12MiB-------------------------------------------------------------------] 0 ∆ 0 Done in 6ms Node Degrees, started 2021-06-08 16:57:10.616+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Relationship --> Relationship 1/1, started 2021-06-08 16:57:10.675+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 1ms RelationshipGroup 1/1, started 2021-06-08 16:57:10.698+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 12ms Node --> Relationship, started 2021-06-08 16:57:10.725+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 12ms Relationship <-- Relationship 1/1, started 2021-06-08 16:57:10.774+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Count groups, started 2021-06-08 16:57:12.030+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Gather, started 2021-06-08 16:57:12.102+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Write, started 2021-06-08 16:57:12.144+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Node --> Group, started 2021-06-08 16:57:12.175+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms Node counts and label index build, started 2021-06-08 16:57:13.624+0000 [*>-----------------------------------------|LABEL INDEX-----------------|COUNT:24.50MiB------]2.17M ∆2.17M Done in 358ms Relationship counts and relationship type index build, started 2021-06-08 16:57:14.019+0000 [*>:??----------------------------------------------------------------------------------------] 0 ∆ 0 Done in 0ms IMPORT DONE in 21s 975ms. Imported: 2162447 nodes 0 relationships 18224200 properties Peak memory usage: 39.12MiB 2021-06-08 16:57:15.678+0000 INFO [o.n.i.b.ImportLogic] Import completed successfully, took 21s 975ms. Imported: 2162447 nodes 0 relationships 18224200 properties 2021-06-08 16:57:18.384+0000 INFO [StoreCopy] Import summary: Copying of 6153114 records took 28 seconds (219754 rec/s). Unused Records 3990667 (64%) Removed Records 0 (0%) 2021-06-08 16:57:18.384+0000 INFO [StoreCopy] ### Extracting schema ### 2021-06-08 16:57:18.384+0000 INFO [StoreCopy] Trying to extract schema... 2021-06-08 16:57:18.551+0000 INFO [StoreCopy] ... found 4 schema definitions. The following can be used to recreate the schema: 2021-06-08 16:57:18.552+0000 INFO [StoreCopy] CALL db.createUniquePropertyConstraint( 'constraint_user_instagramShadowToken', ['User'], ['instagramShadowToken'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} ); CALL db.createUniquePropertyConstraint( 'constraint_user_shadowtoken', ['User'], ['shadowToken'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} ); CALL db.createUniquePropertyConstraint( 'constraint_user_id', ['User'], ['id'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]} ); CALL db.createIndex('index_country_code', ['User'], ['countryCode'], 'native-btree-1.0', {`spatial.cartesian-3d.min`: [-1000000.0, -1000000.0, -1000000.0],`spatial.cartesian.min`: [-1000000.0, -1000000.0],`spatial.wgs-84.min`: [-180.0, -90.0],`spatial.cartesian-3d.max`: [1000000.0, 1000000.0, 1000000.0],`spatial.cartesian.max`: [1000000.0, 1000000.0],`spatial.wgs-84-3d.min`: [-180.0, -90.0, -1000000.0],`spatial.wgs-84-3d.max`: [180.0, 90.0, 1000000.0],`spatial.wgs-84.max`: [180.0, 90.0]}) 2021-06-08 16:57:18.553+0000 INFO [StoreCopy] You have to manually apply the above commands to the database when it is started to recreate the indexes and constraints. The commands are saved to /Users/fyao/softwares/neo4j-enterprise-4.2.4/logs/neo4j-admin-copy-2021-06-08.09.56.48.log as well for reference. Here is a comparison about the space usage before and after the neo4j-admin copy execution. In your Neo4j browser window, you can type :sysinfo , and it will show you the store size of your database. You can see that the store size shrink from 8.67GB to 503.78MB. more details can be found here:https://neo4j.com/docs/operations-manual/current/backup-restore/copy-database/ InformationTitlenodes deleted Vs. nodes physically delete from hard drive (Reclaim unused space)Article Created Date3/15/2024, 10:57 PMURL Name4401993644947-nodes-deleted-Vs-nodes-physically-delete-from-hard-drive-Reclaim-unused-spaceLast Modified Date3/20/2024, 8:55 PM