CRC's and Checksums

[last updated: 2022-10-10]
see also: SHA256
-----

• CRC's and Checksums are algorithms that produce a number that can be used to identify or validate the accuracy of a packet of data.
  Some online sites say that a CRC is a type of checksum, but I find that more semantics than a useful categorization.
  Categorically, IMO, the most reasonable thing to say is that they are both devices/methods to detect errors in data packets.
• Data Packets are often composed of something like this:
  a start byte
  perhaps an ID byte or two
  perhaps a data length byte or two
  the data/message itself
  a stop byte
  a CRC or Checksum byte or bytes

  As you can see, the CRC or Checksum is added to the end of the message/data packet.
  Note: While there are standards, eg. for video or internet file transfer,
  in fact a given user/implementation is free to define the data packet protocol/format that he wants,
  as long as whoever he's communicating with uses the same packet structure, of course...
  ------------------------------------------------------------------

• Create a CRC or Checksum:
  There are a lot of alternative algorithms to create a CRC or checksum.
  Here's a generic procedure:
  • Take all the bytes in the message,
    usually not including the start byte,
    maybe including the ID or length data or not,
    definitely including the data/message itself,
    maybe including the stop byte or not,
    not including the CRC/checksum
  • Put them all through the specific algorithm you desire.
    The output of the algorithm is the checksum or CRC.

• If you are creating a data packet:
  then add the CRC/cksum to the end of the packet.
• If you are validating a packet:
  then compare the CRC/cksum you calculated
  with the one that is actually attached to the packet you received.
  If they match, the data is valid and without error (at least as far as the accuracy of your method is concerned...)
  ------------------------------------------------------------------

• Types:
  • Simple 1's Complement Checksum:
    This method has the advantage of being easy to calculate and verify.
    However it is not as robust at detecting errors as a CRC method.
    • The "1's complement checksum" is the 1's complement of the sum of all the bytes of data:
    • Calculate S = sum of all the bytes
      (excluding start, length, etc. or not as desired)
    • Ignore any carry
      ie. S = S modulo 256
    • Calculate checksum C = 1's complement of S
      ie. convert all 1's to 0's, and all 0's to 1's
      ie. C = 255 - S

    • To validate a packet:
      • With the method above,
        Calculate a checksum for the set that includes all the data plus the received checksum.
        If it equals 0x00, then the packet is valid.

      • -OR- Calculate the 8-bit sum of all the data bytes .
        Add this sum to the checksum in the received packet.
        If the result of the summation is all 1's, then the packet is valid.

      ----------------------------------------------------------

    • CRC's: Cyclic Redundancy Check:
      • CRC's are difficult to calculate "by hand." Officially, the calculation involves:
        the remainder of a polynomial division of the total contents of the file.
      • Luckily there are lots of libraries available that do the calculations for you for whichever algorithm option you desire.
        A few are listed below:
      • crc32 in zlib
      • https://www.libcrc.org/
      • https://github.com/d-bahr/CRCpp
      • https://create.stephan-brumme.com/crc32/
      • https://pypi.org/project/crc/

      -------------------------------------------

  .
  

  .
  

  .
  

  eof