Christian Lemmy
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
sanitation@lemmy.today to Programmer Humor@programming.dev · 2 months ago

New compression technique

lemmy.today

message-square
145
link
fedilink
1

New compression technique

lemmy.today

sanitation@lemmy.today to Programmer Humor@programming.dev · 2 months ago
message-square
145
link
fedilink

alert-triangle
You must log in or # to comment.
  • pressanykeynow@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Could have just stored those in PiFS https://github.com/philipl/pifs

    • lad@programming.dev
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      That’s nice, albeit I want to point out for anyone wondering that this is only conjectured and not guaranteed:

      One of the properties that π is conjectured to have is that it is normal, which is to say that its digits are all distributed evenly, with the implication that it is a disjunctive sequence, meaning that all possible finite sequences of digits will be present somewhere in it.

      There is no guarantee for any specific sequence to appear in π, but for short chunks chances are better (it’s not really a probability, but it’s simpler to say and I can’t explain in details anyway). That’s because (from wiki):

      It is widely believed that the (computable) numbers √2, π, and e are normal, but a proof remains elusive.

    • tetris11@feddit.uk
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      It’s pronounced “Piff Es” for anyone wondering

      • Axolotl@feddit.it
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Idk if it’s irony, but the real pronounce is probably /paɪ ɛf-ɛs/ (english)
        or /'pi fs/ in every other language that reads letters as they are written

  • BilSabab@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Remember that Sloot thing from late 90s?

    • Slashme@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      No, what was it?

      • BilSabab@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        found it - here’s a detailed account https://corecursive.com/sloot-digital-coding-system/

        • ThePuy@feddit.nl
          link
          fedilink
          arrow-up
          0
          ·
          2 months ago

          Wow that was quite the read, thanks!

          • Čauky Mňauky@lemmy.zip
            link
            fedilink
            arrow-up
            0
            ·
            2 months ago

            Yeah, that had me hooked to the screen.

          • BilSabab@lemmy.world
            link
            fedilink
            arrow-up
            0
            ·
            2 months ago

            you’re welcome

        • IsoKiero@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          Very interesting piece of history. Thank you!

          • BilSabab@lemmy.world
            link
            fedilink
            arrow-up
            0
            ·
            2 months ago

            you’re welcome

      • GodSpeeD808@feddit.nl
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Start of the rabbithole

      • BilSabab@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        so basically there was a dutch tech bro in the late 90s who claimed he had developed a compression technique that could turn a movie into just a couple of kilobytes and right when he was going to sign with some investors he mysteriously passed away leaving no documents explaining his invention.

        • BCsven@lemmy.ca
          link
          fedilink
          arrow-up
          0
          ·
          2 months ago

          And his investors family were stabbed, and another guy relates to it had is car blow up. Random or coordinated?

        • khánh@lemmy.zip
          link
          fedilink
          arrow-up
          0
          ·
          2 months ago

          why would someone kill him over it? what do you think?

          • Nalivai@lemmy.world
            link
            fedilink
            arrow-up
            0
            ·
            2 months ago

            Not saying that’s what happened, but if I ever find myself in a similar situation, I’m faking my death and move halfway across the world in heartbit. After I get the money of course

          • BilSabab@lemmy.world
            link
            fedilink
            arrow-up
            0
            ·
            2 months ago

            most likely he just overstressed because he sold folks a bill of goods

            • CanadaPlus@lemmy.sdf.org
              link
              fedilink
              arrow-up
              0
              ·
              2 months ago

              Yup. It seems a lot more likely than solving a problem that we still can’t solve, and that might not even be possible, in the 90’s.

  • Tenderizer@aussie.zone
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I’d be game for that. I hate the fact that photos of me exist in this world.

  • Sanctus@anarchist.nexus
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Lmfao “tears of joy”

    • toy_boat_toy_boat@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Precisely! We human people all know that tears definitely do not express joy.

      Correct?

  • MalReynolds@slrpnk.net
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Trying to find it funny, but in 2026 it’s way too close to the bone and just makes me sad.

    • jafra@slrpnk.net
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      Yeah. I first thought it’s about jpeg, then i read ai and i wasnt sure if its worrying stupidity reporting or bad satire.

    • UnderpantsWeevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Had Sam Altman personally vocalized this article in full sincerity, I would not have been surprised.

    • Maeve @lemmygrad.ml
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      It’s what Apple and MS basically do by uploading all your images to iCloud and showing the name in your images folder.

      Edit to ask: is this what the human brain does when retrieving a memory?

    • AmyAye@nord.pub
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Yeah, like how they are destroying rare books.

      I mean, you don’t get that back just by asking the content theft machine.

  • einkorn@feddit.org
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Throw some angel investment cash at this guy already!

  • JasonDJ@lemmy.zip
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    All fun and games until you post a story about how clumsy your dad was and now all your photos are updated to show him as just a giant thumb…with thumbs for arms, and legs, and hands and feet, and fingers and toes. Just all thumbs, all the way down. Like tripping on acid with Rita Repulsa.

    • OR3X@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      • Hamartia@lemmy.world
        cake
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Dd…ddaadd?

      • spankysalmon@fedinsfw.app
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        All I see is a police man wearing red, I don’t get it.

    • boonhet@sopuli.xyz
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      For some reason all my photos now show me having two left hands.

  • ryannathans@aussie.zone
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Well actually…

    https://www.nature.com/articles/s42256-025-01033-7

    Lossless compression by LLM

    • urushitan 漆たん@kakera.kintsugi.moe
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      Non paywalled

      https://arxiv.org/abs/2407.07723

    • BlackLaZoR@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      I’d be more interested in lossy compression tho. Few % of image quality loss for multiplying compression ratio is mega useful.

    • Mikina@programming.dev
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      Is it a compression when you need presumably gigabites of a model to reconstruct the data?

      It’s like saying rainbow tables are compression.

      • ryannathans@aussie.zone
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        A roughly 340MB model at fp16 is all it takes to beat xz and zstd by 2x compression on text

    • WesternInfidels@feddit.online
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      They’re using domain-specific LLMs to compress narrow-domain data. Their text compression LLM was trained on, and then tested on, legal text and medical text.

      There’s no reason one couldn’t apply the domain-specific-compressor idea to a conventional lossless text compressor, essentially moving much of the dictionary from the compressed file to the program itself. I don’t know if anyone’s tried that. I’d like to know how that compares.

      • ryannathans@aussie.zone
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        There’s a number of compression projects working on this, this is the first that comes to mind

        https://bellard.org/nncp/

        https://bellard.org/ts_zip/

  • OldGrayDog@fedinsfw.app
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I’ve read that the Trump administration is hiring him to archive all of the Epstein files using that format.

    • some_designer_dude@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Compressing the Epstein files is easy: “Donald Trump raped children.” What’s my Weissman score?

  • BlackLaZoR@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    I imagine you could actually make an insane compression algo if you used AI as support for regular jpg compression. Not this way tho.

    • spankysalmon@fedinsfw.app
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Your imagination is not how computer science works

  • bandwidthcrisis@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Whatever happened to fractal image compression, which compressed an image by finding parts similar to larger regions of the picture, so that is could reconstruct the image from itself.

    • BlackLaZoR@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      It only works if the image is actually fractal-like. Real world is a bitch

      • phonics@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Were in a fractal just too zoomed in to notice

      • ChickenLadyLovesLife@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Sometimes the world is a Mandel Bro and sometimes it isn’t.

  • Anchorxiety@reddthat.com
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    I compress my meals by putting them into a blender to liquify them before consumption!

  • AItoothbrush@lemmy.zip
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    The fuuuucking annoying part is that these weight trained models would be perfect for translation models, compression, etc. An llm is already kind of a really efficient lossy compressor but you could actually make it lossless and an actual compressor if used properly. But instead people are literally telling llms to translate instead of training models that are for translating. The technology isnt the problem itself, its the industry and capitalism.

    • spankysalmon@fedinsfw.app
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      lol no you cannot make it lossless.

      • AItoothbrush@lemmy.zip
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Yeah you can? A neural network that only relies on wheights and doesnt use random numbers will always give the same output for the same input.

        • spankysalmon@fedinsfw.app
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          So does lossy compression… same output for input of an algorithm is deterministic, not lossless…

      • TrickDacy@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        Enhance!

    • ChaoticNeutralCzech@feddit.org
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      There are not many places where 95% compression of UTF-8 plain text could outweigh needing a 30GB model in memory and a significant fraction of current LLM inference cost to decompress it − and good luck convincing librarians to adopt it.

      • Natanael@infosec.pub
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        There’s a 300 MB library for audio compression using it. If you have large audio libraries it could eventually become worth the tradeoff.

        https://huggingface.co/facebook/encodec_32khz

        The image versions are probably more useful though.

        The important part for these codes is that it has pre-LLM functions for quality metrics to judge if the output is close enough to indistinguishable (preserves detail, doesn’t add any).

        Although there is also variants deriving a neural net from the media to recreate it from the smaller model.

  • ShellMonkey@piefed.socdojo.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    On a similar note, I saw a story a bit back of someone saving input tokens by feeding the bot an image of a wall of text rather than the text itself and having it read the image via OCR.

    Satire and reality are too hard too distinguish these days.

    • topperharlie@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      I mean, back in my days (👴) people sent a photo pasted into a word document, so things haven’t change that much…

      • Natanael@infosec.pub
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        They still do

    • skisnow@lemmy.ca
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      I see this one all the time on LinkedIn, the management rubes love it

    • SorryQuick@lemmy.ca
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      Saves input tokens and uses more output because of OCR, and output is priced way higher.

      • Rooster326@programming.dev
        link
        fedilink
        arrow-up
        0
        ·
        2 months ago

        It’s an accounting trick. It uses a different budget…

  • PattyMcB@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    This reminds me of IPoAC

    https://en.wikipedia.org/wiki/IP_over_Avian_Carriers

    • fruitycoder@sh.itjust.works
      link
      fedilink
      arrow-up
      0
      ·
      2 months ago

      No no IPoAC is far more viable

Programmer Humor@programming.dev

programmer_humor@programming.dev

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !programmer_humor@programming.dev

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

  • Keep content in english
  • No advertisements
  • Posts must be related to programming or programmer topics
    • If the mod doesn’t find it funny, you’re banned. Ha-ha!.. For real: do not use the community for “statements”. There are other places for such content. Keep it chill and funny.
Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 271 users / day
  • 668 users / week
  • 1.81K users / month
  • 4.16K users / 6 months
  • 1 local subscriber
  • 33.9K subscribers
  • 1.43K Posts
  • 24K Comments
  • Modlog
  • mods:
  • adr1an@programming.dev
  • Feyter@programming.dev
  • BurningTurtle@programming.dev
  • Pierre-Yves Lapersonne@programming.dev
  • UI: unknown version
  • BE: 0.19.19
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org