Skip to content

Latest commit

 

History

History
149 lines (124 loc) · 5.65 KB

lecture.md

File metadata and controls

149 lines (124 loc) · 5.65 KB

Lecture 14

High-level Languages and Operating System.

Lecture

OS and Programming Languages

Linux:

  • Kernel — C
  • Core system libraries — C
  • Core system utilities — mostly C and shell
    • Can be C++, Python and Perl
  • Scripting and integrating
    • Mostly shell, but Perl was popular and Python is now
    • Sometimes C++
  • Almost any OS distribution — ''all'' kinds of programming languages

⇒ Is C the only system programming language?

Windows:

  • Core, kernel and libraries: C++ (seldom C)
  • Scripting and integrating:
    • Any .Net-based platform (mostly C#)
    • C++
    • Powershell (was: cmd)
    • JScript, VBScript (Windows Script Host)
  • Applications: C++, .Net, almost any 3d-party languages

MacOS:

  • Kernel — C
  • Core system and libraries — Objective C and C, seldom C++
  • Scripting and integration: AppleScript and Shell
  • Applications: Swift, Objective C, maybe C++, almost any 3d-party languages

Python and OS Programming

  • Cross-platformness
    • +/-
    • ⇒ own implementation of OS features
  • POSIX-oriented (=> Linux)
    • +/-
    • ⇒ own implementation of non-POSIX OS features
  • High-level, but not OS-oriented (unlike shell)
    • +/-
    • ⇒ own implementation of OS features
  • Has a lot of non-privileged syscalls wrappers
  • Incredible amount of modules at PyPi
    • Including system-oriented
    • Including service-oriented
  • Non «resource scrimp» style
    • If resources do not multiply, use it as a whole, not piece by peice
      • E. g. .read() instead of .readline()
    • no needs to count CPU circles :)
    • …

Modules os ans sys

  • Cross-platform path: os.path and pathlib
    • .is*(), .exists() etc.
  • os:
    • .environ
    • syscalls wrappers (.fork, .getpid, .fstat, .popen, .wait, almost any! ...)
  • sys: .executable, .argv, .stdin, .stdout, .stderr, ...
  • Raising a level example: time → datetime → calendar

Also:

Subprocess

Concept: cross-platform process execution with communication and exit status control.

  • Just run and get a result: run()
    • capture_output/input=; stdin=/stdout=/stderr=
    • ...
  • High-level popen analog: Popen()
    • example 
      from subprocess import *
# Run first process that outputs to the unnamed pipe
p1 = Popen(["cal -s"], stdout=PIPE)
# Run second process that inputs from the other end of the pipe opened
# and outputs to the second pipe
p2 = Popen(["hexdump", "-C"], stdin=p1.stdout, stdout=PIPE)
# Allow p1 to receive a SIGPIPE if p2 exits
p1.stdout.close()
# Read from the second pipe (stdout, stderr),
# but stderr will be empty because no redirection is used
res = p2.communicate()
# Note data is bytes, not str
print(res[0].decode())
    • do not use os.system(), it is platform-dependent and unsafe

Multiprocessing

About multithread programming in Python:

  • It exists
  • It almost non-parallel because of Global_interpreter_lock
    • There is no apparent way to eliminate GIL without significant slowing single-threaded code
  • You can use multithread if:
    1. Only one thread eats CPU, but other ones perform I/O
    2. You have complex code design based on threads and significant amount of permanent usage of joint resources
  • Unlike C programs, Python ones have no actual difference between threaded and multiprocess design
  • Paper about Python GIL

The multiprocessing module:

  • Crossplatformness
    • Linux: fork() is used
  • Child process is running ''a function'' (unlike classic fork(), which defines two equal processes)
    • That simplifies data transfer down to the arguments/return in simple cases
  • Processes can communicate through special socket-like high-level objects or object queue
  • Processes can use high-level shared memory-alike objects or object manager (the last is slower, but can work over network!)
  • Processes can be orchestered into a pool running exactly N processs in parallel.
    1. Exact N child processes (called workers) are started
    2. Each worker can execute a function given multiple times as pool flows
    • No other start/stop actions is performed
    1. Workers are stopped when pool is empty

Workshop

Workshop