In Neo4j, when a node is deleted, it essentially becomes unreferenced, and instead of involving a write operation (that would consume IOPS and involve a system call), it only affects the metadata.

As a result, the datastore size on disk doesn't shrink accordingly but becomes essentially "uncompacted" with many unused disk data space reserved but not used.    
See more information on space reuse.

While Aura does not offer compaction as a feature, you can use Neo4j Desktop to perform an offline compaction of your Aura instance's datastore and upload it back to your instance.

1. Will store compaction be beneficial for me?
2. Is there a risk of data loss? YES. See Cautionary notes / Caveats
3. Pre-requisites
4. Perform the Database Store compaction:
   Quick Summary of steps:
   
   1. Identify the Neo4j version of your Aura instance
   2. Create a new on-demand snapshot of your Aura instance and export it
   3. Create a new local Instance in Neo4j Desktop with the same version as the Aura instance and import the Snapshot (One step)
   4. Create a compacted copy of the neo4j database
   5. Recreate Indexes and constraints
   6. Upload the compacted copy to Neo4j Aura
5. References

 

1. Will store compaction be beneficial for me?

If you have deleted a large amount of data from your graph, store compaction recovers the unused nodeIDs, reducing the storage used.
You can use the 'Storage Used' metric in the Neo4j Aura console to check the trend of storage use.
You could also run the `:sysinfo` procedure from the browser to check the datastore's size.

Below is an exaggerated example where database store compaction would be extremely beneficial

• 14 million nodes with 7 million relationships were initially created in the graph
• Later, 14 million nodes were deleted.
• However, the store size remains unchanged
• The increase in store size is from the additional transaction logs and can be ignored as the transaction logs will be pruned during the next checkpoint.
   
  

2. Cautionary notes

• As Aura does not offer the compaction feature as standard, the steps provided involve exporting a snapshot, performing compaction offline and reimporting the compacted database store.
• Any WRITES performed directly on the Aura instance between the initial export and the final compacted import will be lost.
• It is best to approach this operation as one that requires downtime.
• At the very least, only run read queries on the database between the export and import.
• We cannot estimate the amount of storage that can be reclaimed, as the storage used by a node depends on its properties.
• We will be compacting and replacing only the neo4j database. The system database will be untouched, so Roles and Permissions will remain untouched.
• The compaction step will not copy all indexes. A file containing the commands to recreate them later is generated. 

3. Pre-requisites:

1. Local computer with the latest version of Neo4j Desktop installed
   • It is important to install Neo4j Desktop version 2, as version 1.x does not support Neo4j Calver, used by Aura instances. The latest version of Neo4j Desktop, as of this article being written, is 2.1.4
2. The available storage capacity on your computer should be at least three times the size of your Aura instance's Total Store Size.
3. This is to accommodate: 
   1. The snapshot file originally downloaded from the Aura instance
   2. The local database created with this dump
   3. Compacted database created 
   4. Dump file created from the compacted database
      Run the :sysinfo command from the Neo4j Browser on your Aura instance to find the instance's Total Store Size.

4. Perform the Database Store compaction:

1. Identify the Neo4j version of your Aura instance on the Aura console, click the three dots (more instance actions) and then select Inspect. The Overview tab will display information about your instance, including the version. 
     
2. Create a new on-demand snapshot of your Aura instance and export it.
   Download and save it locally. You should have a file like neo4j-<yyyy-mm-dd-timestamp-id>.backup
   As mentioned, any changes made to your Aura instance after this point will be lost at the end of the import done finally.
   Hence, it is advisable not to make any WRITEs to your instance.
   If possible, it may be better to pause your instance at this point.
3. Create a new local instance in Neo4j Desktop and import the Snapshot:
   1. Name the instance as desired
   2. Select the same Neo4j version as your Aura instance
   3. Add database user details
   4. Use the 'Browse' button under 'Load from .dump, .backup or .tar (optional)' and select the file downloaded in the previous step
   5. Click Create
       
      
4. Click 'Open Folder' to open the path for the newly created local database in your computer's file explorer
5. Open the terminal in your computer for this path. The steps for this would vary based on your operating system.
   In MacOS, you can select the path and click 'Open in Terminal'

6. Create a compacted copy of the neo4j database, called neo4jCompressedCopy:
   The syntax provided is for MacOS and would work for other Unix-like systems as well
   For Windows Systems, replace forward slashes  '/' with  backslashes - '\'
   Unix-like Systems (Linux, Unix and MacOS):
   bin/neo4j-admin database copy neo4j neo4jCompressedCopy --compact-node-store

   MS Windows: 
   bin\neo4j-admin database copy neo4j neo4jCompressedCopy --compact-node-store
   
   If your datastore is large you need to help allocate more memory to process large chunks at a time by adding the following flags to the `neo4j-admin copy` command (subject to availability of Memory in your local computer):
   --to-pagecache=4G
--from-pagecache=16G (as close as possible to the source datastore size)
    
7. Make a note of the indexes to be recreated, using the cypher displayed on the screen or the `neo4jcompressedcopy-schema.cypher` file generated in the same folder at the end of the database copy
    
    
8. Start the local DBMS from the Neo4j Desktop and Open it with Neo4j Browser
9. Run CREATE DATABASE neo4jCompressedCopy  to create the copied database
10. Validate it started and that the compaction operation worked successfully.
    1. Run SHOW DATABASES to check it is online.
    2. Run :use neo4jCompressedCopy  to switch to it and run :sysinfo  to check the new Store size and ID allocation
    3. Run cypher queries to get a summary of the graph . Compare it with your Aura instance if unsure.
    4. Any other way you can verify your data's integrity
       Note: No indexes would have been copied. 
       The store size has reduced drastically.
       
11. Recreate the Indexes and Constraints:
    1. You could either drag and drop the file created in the previous step into the Query window and execute it
    2. Or open the file manually using a text editor, copy the commands and executre them manually
        
       
12. Navigate back to the 'Local Instances' tab and stop the instance
     
13. You are now ready to upload the compressed copy back into your Aura instance. If you had previously paused your Aura instance, resume it from the Aura Console and wait for it to return to a 'Running' state.
14. Click the Aura drop-down and select 'Deploy to Neo4j Aura'
     
15. Select the compressed DB and click Continue
     
16. Enter the Aura instance's connection details and credentials and select 'Overwrite and Continue'
    THIS WILL OVERWRITE YOUR AURA INSTANCE, AND YOU WILL LOSE ANY DATA CHANGES DONE TO THE INSTANCE SINCE THE ON-DEMAND SNAPSHOT WAS TAKEN (Step 4.2)
    
    
    
17. You can monitor the upload's progress in Neo4j Desktop or from the Aura Console.
    In the Aura Console, the instance will go From 'Running' to 'Loading' and then back to 'Running' again
     
18. Once the upload is complete, login to the instance and verify data integrity and check the store size using :sysinfo or the metrics from teh Aura Console

5. References

• Space reuse
• 'Storage Size' metric in the Neo4j Aura console
• Neo4j Desktop