A Federation
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
Séra Balázs@lemmy.world to linuxmemes@lemmy.world · 9 months ago

Remember to not to forget clearing your journal

lemmy.world

message-square
62
fedilink
666

Remember to not to forget clearing your journal

lemmy.world

Séra Balázs@lemmy.world to linuxmemes@lemmy.world · 9 months ago
message-square
62
fedilink
alert-triangle
You must log in or register to comment.
  • Morethanevil@lemmy.fedifriends.social
    link
    fedilink
    arrow-up
    153
    ·
    9 months ago

    Cleanup

    Check current disk usage:

    sudo journalctl --disk-usage

    Use rotate function:

    sudo journalctl --rotate

    Or

    Remove all logs and keep the last 2 days:

    sudo journalctl --vacuum-time=2days

    Or

    Remove all logs and only keep the last 100MB:

    sudo journalctl --vacuum-size=100M

    How to read logs:

    Follow specific log for a service:

    sudo journalctl -fu SERVICE

    Show extended log info and print the last lines of a service:

    sudo journalctl -xeu SERVICE

    • macniel@feddit.org
      link
      fedilink
      arrow-up
      39
      ·
      9 months ago

      I mean yeah -fu stands for “follow unit” but its also a nice coincidence when it comes to debugging that particular service.

      • Morethanevil@lemmy.fedifriends.social
        link
        fedilink
        arrow-up
        4
        ·
        9 months ago

        😂😂

    • lseif@sopuli.xyz
      link
      fedilink
      arrow-up
      24
      ·
      9 months ago

      --vacuum-time=2days

      this implies i keep an operating system installed for that long

      • jbk@discuss.tchncs.de
        link
        fedilink
        arrow-up
        9
        ·
        9 months ago

        something something nix?

    • slazer2au@lemmy.world
      link
      fedilink
      English
      arrow-up
      19
      ·
      9 months ago

      sudo journalctl --disk-usage

      panda@Panda:~$ sudo journalctl --disk-usage  
      No journal files were found.  
      Archived and active journals take up 0B in the file system.
      

      hmmmmmm…

    • superkret@feddit.org
      link
      fedilink
      arrow-up
      12
      ·
      edit-2
      9 months ago
      user@u9310x-Slack:~$ sudo journalctl --disk-usage  
      Password:  
      sudo: journalctl: command not found  
      
      • macniel@feddit.org
        link
        fedilink
        arrow-up
        20
        ·
        edit-2
        9 months ago

        seems like someone doesn’t like systemd :)

        • superkret@feddit.org
          link
          fedilink
          arrow-up
          5
          ·
          9 months ago

          I don’t have any feelings towards particular init systems.

          • Scipitie@lemmy.dbzer0.com
            link
            fedilink
            arrow-up
            4
            ·
            9 months ago

            Just curious, what distro do you use that systemd is not the default? (I at least you didn’t change it after the fact if you don’t have any feelings (towards unit systems ;) ) )

            • superkret@feddit.org
              link
              fedilink
              arrow-up
              5
              ·
              9 months ago

              Slackware

    • kralk@lemm.ee
      link
      fedilink
      arrow-up
      9
      ·
      9 months ago

      Badass! Thanks!

    • Tekkip20@lemmy.world
      link
      fedilink
      arrow-up
      8
      ·
      9 months ago

      Thank you for this, wise sage.

      Your wisdom will be passed down the family line for generations about managing machine logs.

      • Morethanevil@lemmy.fedifriends.social
        link
        fedilink
        arrow-up
        5
        ·
        9 months ago

        Glad to help your family, share this wisdom with friends too ☝🏻😃

      • VirtualOdour@sh.itjust.works
        link
        fedilink
        arrow-up
        2
        ·
        9 months ago

        Yeah, if I had dependents they’d gather round the campfire chanting these mystical runes in the husk of our fallen society

    • elxeno@lemm.ee
      link
      fedilink
      arrow-up
      6
      ·
      9 months ago

      @RemindMe@programming.dev 6 months

      • elxeno@lemm.ee
        link
        fedilink
        arrow-up
        2
        ·
        9 months ago

        @ategon@programming.dev is the remindme bot offline?

        • Ategon@programming.dev
          link
          fedilink
          arrow-up
          3
          ·
          edit-2
          9 months ago

          Its semi broken currently and also functions on a whitelist with this community not being on the whitelist

          • elxeno@lemm.ee
            link
            fedilink
            arrow-up
            1
            ·
            9 months ago

            Ok, thanks!

    • Pacmanlives@lemmy.world
      link
      fedilink
      arrow-up
      3
      ·
      9 months ago

      Actually something I never dug into. But does logrotate no longer work? I have a bunch of disk space these days so I would not notice large log files

      • Morethanevil@lemmy.fedifriends.social
        link
        fedilink
        arrow-up
        2
        ·
        edit-2
        9 months ago

        If logrotate doesn’t work, than use this as a cronjob via sudo crontab -e Put this line at the end of the file:

        0 0 * * * journalctl --vacuum-size=1G >/dev/null 2>&1

        Everyday the logs will be trimmed to 1GB. Usually the logs are trimmed automatically at 4GB, but sometimes this does not work

        • fallingcats@discuss.tchncs.de
          link
          fedilink
          arrow-up
          5
          ·
          9 months ago

          If we’re using systemd already, why not a timer?

          • Morethanevil@lemmy.fedifriends.social
            link
            fedilink
            arrow-up
            3
            ·
            9 months ago

            Cron is better known than a systemd timer, but you can provide an example for the timer 😃

            • fallingcats@discuss.tchncs.de
              link
              fedilink
              arrow-up
              5
              ·
              edit-2
              9 months ago

              Really, the correct way would be to set the limit you want for journald. Put this into /etc/systemd/journald.conf.d/00-journal-size.conf:

              [Journal]
              SystemMaxUse=50M
              

              Or something like this using a timer: systemd-run --timer-property=OnCalender=daily $COMMAND

              • Morethanevil@lemmy.fedifriends.social
                link
                fedilink
                arrow-up
                3
                ·
                9 months ago

                Thanks for this addition ☺️

    • rambling_lunatic@sh.itjust.works
      link
      fedilink
      arrow-up
      3
      ·
      9 months ago

      If you use OpenRC you can just delete a couple files

    • MonkderVierte@lemmy.ml
      link
      fedilink
      arrow-up
      2
      ·
      9 months ago

      Why isn’t it configured like that by default?

      • faerbit@sh.itjust.works
        link
        fedilink
        arrow-up
        5
        ·
        9 months ago

        It is. The defaults are a little bit more lenient, but it shouldn’t gobble up 80 GB of storage.

      • Morethanevil@lemmy.fedifriends.social
        link
        fedilink
        arrow-up
        2
        ·
        9 months ago

        Good question, it may depend on the distro afaik

  • RobotZap10000@feddit.nl
    link
    fedilink
    arrow-up
    31
    ·
    9 months ago

    Try 60GB of system logs after 15 minutes of use. My old laptop’s wifi card worked just fine, but spammed the error log with some corrected error. Adding pci=noaer to grub config fixed it.

    • xilophor@lemmy.world
      link
      fedilink
      English
      arrow-up
      24
      ·
      9 months ago

      I had an issue on my PC (assuming faulty graphics driver or bug after waking from sleep) that caused my syslog file to reach 500GiB. Yes, 500GiB.

      • PenisDuckCuck9001@lemmynsfw.com
        link
        fedilink
        arrow-up
        10
        ·
        edit-2
        9 months ago

        Nearly 700gb in logs

        wtf 🤯

        • xilophor@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          9 months ago

          Yeah I was confused as to where all of my storage went xD

  • Andrew@mander.xyz
    link
    fedilink
    arrow-up
    35
    arrow-down
    5
    ·
    edit-2
    9 months ago

    *cough*80 GiB*cough*

    • Ironfacebuster@lemmy.world
      link
      fedilink
      arrow-up
      5
      ·
      9 months ago

      11.6 mega bites

      • Andrew@mander.xyz
        link
        fedilink
        arrow-up
        5
        ·
        9 months ago

        Ah, yes, the standard burger size.

  • Possibly linux@lemmy.zip
    link
    fedilink
    English
    arrow-up
    30
    ·
    9 months ago

    You just need a bigger drive. Don’t delete anything

    • b00m@lemmy.world
      link
      fedilink
      arrow-up
      10
      ·
      9 months ago

      Oh lord watch me hoard

  • hushable@lemmy.world
    link
    fedilink
    arrow-up
    26
    ·
    edit-2
    9 months ago

    Once I had a mission critical service crash because the disk got full, turns out there was a typo on the logrotate config and as a result the logs were not being cleaned up at all.

    edit: I should add that I used the commands shared in this post to free up space and bring the service back up

  • wildbus8979@sh.itjust.works
    link
    fedilink
    arrow-up
    28
    arrow-down
    3
    ·
    edit-2
    9 months ago

    Fucking blows my mind that journald broke what is essentially the default behavior of every distro’s use of logrotate and no one bats an eye.

    • Regalia@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      26
      arrow-down
      1
      ·
      edit-2
      9 months ago

      I’m not sure if you’re joking or not, but the behavior of journald is fairly dynamic and can be configured to an obnoxious degree, including compression and sealing.

      By default, the size limit is 4GB:

      SystemMaxUse= and RuntimeMaxUse= control how much disk space the journal may use up at most. SystemKeepFree= and RuntimeKeepFree= control how much disk space systemd-journald shall leave free for other uses. systemd-journald will respect both limits and use the smaller of the two values.

      The first pair defaults to 10% and the second to 15% of the size of the respective file system, but each value is capped to 4G.

      • Max-P@lemmy.max-p.me
        link
        fedilink
        arrow-up
        13
        ·
        9 months ago

        If anything I tend to have the opposite problem: whoops I forgot to set up logrotate for this log file I set up 6 months ago and now my disk is completely full. Never happens for stuff that goes to journald.

      • wildbus8979@sh.itjust.works
        link
        fedilink
        arrow-up
        6
        arrow-down
        2
        ·
        edit-2
        9 months ago

        It can be, but the defaults are freaking stupid and often do not work.

        • Starbuck@lemmy.world
          link
          fedilink
          arrow-up
          19
          arrow-down
          1
          ·
          9 months ago

          Aren’t the defaults set by your distro?

          • wildbus8979@sh.itjust.works
            link
            fedilink
            arrow-up
            2
            ·
            9 months ago

            AAfaict Debian uses the upstream defaults.

    • tentacles9999@lemmynsfw.com
      link
      fedilink
      English
      arrow-up
      5
      ·
      9 months ago

      Still boggles my mind that systemd being terrible is still a debate. Like of all things, wouldn’t text logs make sense?

      • acockworkorange@mander.xyz
        link
        fedilink
        arrow-up
        5
        ·
        9 months ago

        Wait… it doesn’t store them in plaintext?

        • uranibaba@lemmy.world
          link
          fedilink
          arrow-up
          4
          ·
          9 months ago

          Nope, you need journalctl to read.

          • acockworkorange@mander.xyz
            link
            fedilink
            arrow-up
            2
            ·
            9 months ago

            That’s asinine.

        • tentacles9999@lemmynsfw.com
          link
          fedilink
          English
          arrow-up
          3
          ·
          9 months ago

          Yeah, and you need systemd to read the binary logs. Though I think there may be a setting to change to text logs, I am not sure because I avoid systemd when I can

      • fallingcats@discuss.tchncs.de
        link
        fedilink
        arrow-up
        4
        ·
        9 months ago

        Wouldn’t compressed logs make even more sense (they way they’re now)?

  • muhyb@programming.dev
    link
    fedilink
    arrow-up
    16
    ·
    9 months ago

    This once happened to me on my pi-hole. It’s an old netbook with 250 GB HDD. Pi-hole stopped working and I checked the netbook. There was a 242 GB log file. :)

  • SuperIce@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    ·
    9 months ago

    Systems/Journald keeps 4GB of logs stored by default.

  • zoey@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    11
    ·
    9 months ago

    Recently had the jellyfin log directory take up 200GB, checked the forums and saw someone with the same problem but 1TB instead.

    • Agent641@lemmy.world
      link
      fedilink
      arrow-up
      17
      ·
      edit-2
      9 months ago

      2024-03-28 16:37:12:017 - Everythings fine

      2024-03-28 16:37:12:016 - Everythings fine

      2024-03-28 16:37:12:015 - Everythings fine

  • Muscar@discuss.online
    link
    fedilink
    arrow-up
    10
    ·
    9 months ago

    Title gore

  • MonkderVierte@lemmy.ml
    link
    fedilink
    arrow-up
    7
    ·
    edit-2
    9 months ago

    logrotate is a thing.

  • Scribbd@feddit.nl
    link
    fedilink
    arrow-up
    6
    ·
    9 months ago

    I recently discovered the company I work for, has an S3 bucket with network flow logs of several TB. It contains all network activity if the past 8 years.

    Not because we needed it. No, the lifecycle policy wasn’t configured correctly.

  • FiniteBanjo@lemmy.today
    link
    fedilink
    arrow-up
    6
    ·
    9 months ago

    Windows isn’t great by any means but I do like the way they have the Event Viewer layout sorted to my tastes.

    • MonkeMischief@lemmy.today
      link
      fedilink
      arrow-up
      3
      ·
      9 months ago

      True that. Sure, I need to keep my non-professional home sysadmin skills sharp and enjoy getting good at these things, but I wouldn’t mind a better GUI journal reader / configurator thing. KDE has a halfway decent log viewer.

      It might also go a long way towards helping the less sysadmin-for-fun-inclined types troubleshoot.

      Maybe there is one and I just haven’t checked. XD

  • alien@lemm.ee
    link
    fedilink
    arrow-up
    5
    ·
    9 months ago

    I couldn’t tell for a solid minute if the title was telling me to clear the journal or not

  • GustavoM@lemmy.world
    link
    fedilink
    English
    arrow-up
    1
    ·
    9 months ago

    Eh, I just set $ROOTFS to ro and my $HOME to rw.

linuxmemes@lemmy.world

linuxmemes@lemmy.world

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !linuxmemes@lemmy.world

Hint: :q!


Sister communities:
  • !tech_memes@lemmy.world
  • !memes@lemmy.world
  • !lemmyshitpost@lemmy.world
  • !risa@startrek.website

Community rules (click to expand)

1. Follow the site-wide rules
  • Instance-wide TOS: https://legal.lemmy.world/tos/
  • Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
  • Understand the difference between a joke and an insult.
  • Do not harrass or attack users for any reason. This includes using blanket terms, like “every user of thing”.
  • Don’t get baited into back-and-forth insults. We are not animals.
  • Leave remarks of “peasantry” to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
  • Bigotry will not be tolerated.
3. Post Linux-related content
  • Including Unix and BSD.
  • Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of sudo in Windows.
  • No porn, no politics, no trolling or ragebaiting.
4. No recent reposts
  • Everybody uses Arch btw, can’t quit Vim, <loves/tolerates/hates> systemd, and wants to interject for a moment. You can stop now.
5. 🇬🇧 Language/язык/Sprache
  • This is primarily an English-speaking community. 🇬🇧🇦🇺🇺🇸
  • Comments written in other languages are allowed.
  • The substance of a post should be comprehensible for people who only speak English.
  • Titles and post bodies written in other languages will be allowed, but only as long as the above rule is observed.
6. (NEW!) Regarding public figures

We all have our opinions, and certain public figures can be divisive. Keep in mind that this is a community for memes and light-hearted fun, not for airing grievances or leveling accusations.

  • Keep discussions polite and free of disparagement.
  • We are never in possession of all of the facts. Defamatory comments will not be tolerated.
  • Discussions that get too heated will be locked and offending comments removed.

 

Please report posts and comments that break these rules!


Important: never execute code or follow advice that you don’t understand or can’t verify, especially here. The word of the day is credibility. This is a meme community – even the most helpful comments might just be shitposts that can damage your system. Be aware, be smart, don’t remove France.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 1.19K users / day
  • 5.5K users / week
  • 9.82K users / month
  • 19.1K users / 6 months
  • 1 local subscriber
  • 24.9K subscribers
  • 1.56K Posts
  • 77.5K Comments
  • Modlog
  • mods:
  • Kevin@lemmy.world
  • zephyr@lemmy.world
  • rtxn@lemmy.world
  • BE: 0.19.8
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org