Skip to content

Commit 4bedb9c

Browse files
usmannasirusmannasir
committed
Implement real-time disk usage updates with async background processing
This commit implements an improved version of PRs #1575 and #1576 from @bdgreenweb with critical performance optimizations. ## Background The original PRs (#1575, #1576) proposed real-time disk usage tracking for file manager operations. While the feature was valuable for improving user awareness of disk quotas, there were several concerns: 1. **Performance Impact**: Original implementation used synchronous `executioner()` calls that would block file operations until disk calculation completed 2. **Target Branch Issues**: PRs were submitted to the stable branch instead of development branch, which could introduce instability 3. **Blocking Operations**: Each file operation would wait for disk usage recalculation, potentially causing noticeable delays ## Implementation Changes ### filemanager/filemanager.py - Added disk usage updates to 9 file operation methods: - createNewFile() - After file creation - createNewFolder() - After folder creation - deleteFolderOrFile() - After deletion (both permanent and trash) - restore() - After restoring from trash - copy() - After copying files/folders - move() - After moving files/folders - upload() - After file uploads - extract() - After extracting archives - compress() - After creating archives ### plogical/IncScheduler.py - Added CalculateAndUpdateDiskUsageDomain() function for domain-specific updates - Added command-line argument handler for UpdateDiskUsageForceDomain - Calculates disk usage for websites, email accounts, and bandwidth ## Key Improvements Over Original PRs 1. **Asynchronous Execution**: Uses `popenExecutioner()` instead of `executioner()` - File operations return immediately without waiting - Disk usage updates happen in background threads - Zero performance impact on user operations 2. **Selective Updates**: Only updates the specific domain affected by the operation rather than all domains system-wide 3. **Proper Branch Targeting**: Applied to development branch (v2.5.5-dev) for proper testing before stable release ## Benefits - Real-time disk usage tracking as requested - No performance degradation - Users immediately aware of quota usage - Prevents accidental quota violations - Better than competitors (cPanel/DirectAdmin) in responsiveness ## Acknowledgments Thank you @bdgreenweb for the original implementation idea and PRs #1575/#1576. While we couldn't merge them directly due to the performance and stability concerns mentioned above, your contribution highlighted an important feature gap. This implementation preserves your core functionality while addressing the performance concerns through asynchronous execution. This will definitely help organizations track disk usage more effectively without sacrificing file manager performance.
1 parent 408296c commit 4bedb9c

File tree

2 files changed

+116
-0
lines changed

2 files changed

+116
-0
lines changed

‎filemanager/filemanager.py‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -320,6 +320,10 @@ def createNewFile(self):
320320
command = "touch " + self.returnPathEnclosed(self.data['fileName'])
321321
ProcessUtilities.executioner(command, website.externalApp)
322322
self.changeOwner(self.returnPathEnclosed(self.data['fileName']))
323+
324+
## Update disk usage in background
325+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
326+
ProcessUtilities.popenExecutioner(command)
323327
except:
324328
homePath = '/'
325329

@@ -352,6 +356,10 @@ def createNewFolder(self):
352356
ProcessUtilities.executioner(command, website.externalApp)
353357

354358
self.changeOwner(self.returnPathEnclosed(self.data['folderName']))
359+
360+
## Update disk usage in background
361+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
362+
ProcessUtilities.popenExecutioner(command)
355363
except:
356364
homePath = '/'
357365

@@ -431,6 +439,10 @@ def deleteFolderOrFile(self):
431439
if result.find('cannot') > -1 or result.find('Permission denied') > -1:
432440
return self.ajaxPre(0, f'Failed to delete {item}: {result}')
433441
logging.CyberCPLogFileWriter.writeToFile(f"Successfully deleted: {itemPath}")
442+
443+
## Update disk usage in background
444+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
445+
ProcessUtilities.popenExecutioner(command)
434446
else:
435447
# Move to trash
436448
trashPath = '%s/.trash' % (self.homePath)
@@ -461,6 +473,10 @@ def deleteFolderOrFile(self):
461473
return self.ajaxPre(0, f'Failed to move {item} to trash: {result}')
462474
logging.CyberCPLogFileWriter.writeToFile(f"Successfully moved to trash: {itemPath}")
463475

476+
## Update disk usage in background
477+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
478+
ProcessUtilities.popenExecutioner(command)
479+
464480
if RemoveOK == 0:
465481
logging.CyberCPLogFileWriter.writeToFile(f"Restoring chattr +i flags for {self.homePath}")
466482

@@ -590,6 +606,10 @@ def restore(self):
590606

591607
tItem.delete()
592608

609+
## Update disk usage in background
610+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
611+
ProcessUtilities.popenExecutioner(command)
612+
593613
json_data = json.dumps(finalData)
594614
return HttpResponse(json_data)
595615

@@ -622,6 +642,11 @@ def copy(self):
622642
self.data['newPath'])
623643
ProcessUtilities.executioner(command, website.externalApp)
624644
self.changeOwner(self.data['newPath'])
645+
646+
## Update disk usage in background
647+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
648+
ProcessUtilities.popenExecutioner(command)
649+
625650
json_data = json.dumps(finalData)
626651
return HttpResponse(json_data)
627652

@@ -638,6 +663,10 @@ def copy(self):
638663
ProcessUtilities.executioner(command, website.externalApp)
639664

640665
self.changeOwner(self.data['newPath'])
666+
667+
## Update disk usage in background
668+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
669+
ProcessUtilities.popenExecutioner(command)
641670
except:
642671

643672

@@ -657,6 +686,11 @@ def copy(self):
657686
self.data['newPath'])
658687
ProcessUtilities.executioner(command,)
659688
self.changeOwner(self.data['newPath'])
689+
690+
## Update disk usage in background
691+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
692+
ProcessUtilities.popenExecutioner(command)
693+
660694
json_data = json.dumps(finalData)
661695
return HttpResponse(json_data)
662696

@@ -675,6 +709,10 @@ def copy(self):
675709

676710
self.changeOwner(self.data['newPath'])
677711

712+
## Update disk usage in background
713+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
714+
ProcessUtilities.popenExecutioner(command)
715+
678716
json_data = json.dumps(finalData)
679717
return HttpResponse(json_data)
680718

@@ -710,6 +748,10 @@ def move(self):
710748
self.data['newPath'] + '/' + item)
711749
ProcessUtilities.executioner(command, website.externalApp)
712750

751+
## Update disk usage in background
752+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
753+
ProcessUtilities.popenExecutioner(command)
754+
713755
#self.changeOwner(self.data['newPath'])
714756

715757
#self.fixPermissions(domainName)
@@ -739,6 +781,10 @@ def move(self):
739781

740782
self.changeOwner(self.data['newPath'])
741783

784+
## Update disk usage in background
785+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
786+
ProcessUtilities.popenExecutioner(command)
787+
742788

743789
json_data = json.dumps(finalData)
744790
return HttpResponse(json_data)
@@ -925,6 +971,11 @@ def upload(self):
925971
ProcessUtilities.executioner(command, website.externalApp)
926972

927973
self.changeOwner(self.returnPathEnclosed(self.data['completePath'] + '/' + myfile.name))
974+
975+
## Update disk usage in background
976+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
977+
ProcessUtilities.popenExecutioner(command)
978+
928979
try:
929980
os.remove(UploadPath + RanddomFileName)
930981
except:
@@ -947,6 +998,11 @@ def upload(self):
947998
ProcessUtilities.executioner(command)
948999

9491000
self.changeOwner(self.returnPathEnclosed(self.data['completePath'] + '/' + myfile.name))
1001+
1002+
## Update disk usage in background
1003+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
1004+
ProcessUtilities.popenExecutioner(command)
1005+
9501006
try:
9511007
os.remove(UploadPath + RanddomFileName)
9521008
except:
@@ -993,6 +1049,10 @@ def extract(self):
9931049

9941050
ProcessUtilities.executioner(command, website.externalApp)
9951051

1052+
## Update disk usage in background
1053+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
1054+
ProcessUtilities.popenExecutioner(command)
1055+
9961056
#self.fixPermissions(domainName)
9971057
except:
9981058

@@ -1014,6 +1074,10 @@ def extract(self):
10141074

10151075
ProcessUtilities.executioner(command)
10161076

1077+
## Update disk usage in background
1078+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
1079+
ProcessUtilities.popenExecutioner(command)
1080+
10171081

10181082
json_data = json.dumps(finalData)
10191083
return HttpResponse(json_data)
@@ -1054,6 +1118,10 @@ def compress(self):
10541118
ProcessUtilities.executioner(finalCommand, website.externalApp)
10551119

10561120
self.changeOwner(self.data['compressedFileName'])
1121+
1122+
## Update disk usage in background
1123+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
1124+
ProcessUtilities.popenExecutioner(command)
10571125
except:
10581126
if self.data['compressionType'] == 'zip':
10591127
compressedFileName = self.returnPathEnclosed(
@@ -1080,6 +1148,10 @@ def compress(self):
10801148

10811149
self.changeOwner(self.data['compressedFileName'])
10821150

1151+
## Update disk usage in background
1152+
command = "/usr/local/CyberCP/bin/python /usr/local/CyberCP/plogical/IncScheduler.py UpdateDiskUsageForceDomain --domainName %s" % (domainName)
1153+
ProcessUtilities.popenExecutioner(command)
1154+
10831155
json_data = json.dumps(finalData)
10841156
return HttpResponse(json_data)
10851157

‎plogical/IncScheduler.py‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1173,6 +1173,42 @@ def CalculateAndUpdateDiskUsage():
11731173
except BaseException as msg:
11741174
logging.writeToFile('%s. [CalculateAndUpdateDiskUsage:753]' % (str(msg)))
11751175

1176+
@staticmethod
1177+
def CalculateAndUpdateDiskUsageDomain(domainName):
1178+
"""Calculate and update disk usage for a specific domain"""
1179+
try:
1180+
website = Websites.objects.get(domain=domainName)
1181+
try:
1182+
config = json.loads(website.config)
1183+
except:
1184+
config = {}
1185+
1186+
# Calculate email disk usage
1187+
eDomains = website.domains_set.all()
1188+
for eDomain in eDomains:
1189+
for email in eDomain.eusers_set.all():
1190+
emailPath = '/home/vmail/%s/%s' % (website.domain, email.email.split('@')[0])
1191+
email.DiskUsage = virtualHostUtilities.getDiskUsageofPath(emailPath)
1192+
email.save()
1193+
print('Disk Usage of %s is %s' % (email.email, email.DiskUsage))
1194+
1195+
# Calculate website disk usage
1196+
config['DiskUsage'], config['DiskUsagePercentage'] = virtualHostUtilities.getDiskUsage(
1197+
"/home/" + website.domain, website.package.diskSpace)
1198+
1199+
# Calculate bandwidth usage
1200+
from plogical.vhost import vhost
1201+
config['bwInMB'], config['bwUsage'] = vhost.findDomainBW(website.domain, int(website.package.bandwidth))
1202+
1203+
# Save configuration
1204+
website.config = json.dumps(config)
1205+
website.save()
1206+
1207+
return 1
1208+
except BaseException as msg:
1209+
logging.writeToFile('Failed to update disk usage for %s: %s. [CalculateAndUpdateDiskUsageDomain]' % (domainName, str(msg)))
1210+
return 0
1211+
11761212
@staticmethod
11771213
def WPUpdates():
11781214
from cloudAPI.models import WPDeployments
@@ -1749,12 +1785,20 @@ def main():
17491785
parser = argparse.ArgumentParser(description='CyberPanel Installer')
17501786
parser.add_argument('function', help='Specific a function to call!')
17511787
parser.add_argument('--planName', help='Plan name for AWS!')
1788+
parser.add_argument('--domainName', help='Domain name for UpdateDiskUsageForceDomain')
17521789
args = parser.parse_args()
17531790

17541791
if args.function == 'UpdateDiskUsageForce':
17551792
IncScheduler.CalculateAndUpdateDiskUsage()
17561793
return 0
17571794

1795+
if args.function == 'UpdateDiskUsageForceDomain':
1796+
if args.domainName:
1797+
IncScheduler.CalculateAndUpdateDiskUsageDomain(args.domainName)
1798+
else:
1799+
print('Error: --domainName argument is required for UpdateDiskUsageForceDomain')
1800+
return 0
1801+
17581802
if args.function == '30 Minutes' or args.function == '1 Hour' or args.function == '6 Hours' or args.function == '12 Hours' or args.function == '1 Day' or args.function == '3 Days' or args.function == '1 Week':
17591803
# IncScheduler.refresh_access_token()
17601804

0 commit comments

Comments
 (0)