I don‘t know if this is the right place to ask, but is this an OK build for a first start in selfhosting? Is there anything obvious I could improve? I‘d mainly run Nextcloud, Immich, some Docker containers and smaller web applications and cronjobs.

  • melroy@kbin.melroy.org
    link
    fedilink
    arrow-up
    1
    ·
    2 days ago
    • I run 50+ websites
    • MariaDB instances
    • PostgreSQL instances
    • Docker containers
    • Mbin, Nextcloud, GitLab, GitLab runners, gitea, bitcoind, fulcrum, grafana, prometheus, influxdb, Synapse, Angie, telegraf and various other services like fail2ban etc. etc.

    All optimized for performance and fine tuned as well, eg. lets say you run mariadb vs how I run it:

    innodb_buffer_pool_size = 8G
    innodb_flush_log_at_trx_commit = 2
    innodb_log_file_size = 2G
    innodb_log_buffer_size = 32M
    innodb_max_dirty_pages_pct = 90
    innodb_io_capacity=5000
    innodb_io_capacity_max=20000
    innodb_read_io_threads=8
    innodb_write_io_threads=8
    query_cache_type = 1
    query_cache_limit = 2M
    query_cache_min_res_unit = 2k
    query_cache_size = 128M
    tmp_table_size= 128M
    max_heap_table_size= 128M
    
    [mysqld]
    max_connections = 200
    character_set_server = utf8mb4
    collation_server = utf8mb4_general_ci
    transaction_isolation = READ-COMMITTED
    binlog_format = ROW
    innodb_file_per_table=1
    # Increase open files based limits.conf value
    open_files_limit=65535
    

    Same idea for Postgresql… You can run “postgres” or… actually run postgresql in production correctly like:

    shared_buffers = 6GB
    work_mem = 20MB
    maintenance_work_mem = 2GB
    maintenance_io_concurrency = 200
    max_worker_processes = 14 
    max_parallel_workers_per_gather = 4
    max_parallel_maintenance_workers = 4
    max_parallel_workers = 12
    
    synchronous_commit = off
    commit_delay = 300
    
    checkpoint_timeout = 30min
    max_wal_size = 60GB
    min_wal_size = 4GB
    
    • hirihit640@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 day ago

      Actually now that I check your username I think I have seen your instances before, though it seems like your website is down at the moment. But thank you for service to open source !

          • melroy@kbin.melroy.org
            link
            fedilink
            arrow-up
            1
            ·
            12 hours ago

            I block some misbehaving data centers. Which are often also used by VPN providers. So it’s more collateral damage.

            No I fully block some data centers not just rate limit.

            • hirihit640@sh.itjust.works
              link
              fedilink
              English
              arrow-up
              2
              ·
              9 hours ago

              Unfortunate but understandable. I believe there are public lists of the IPs of major VPN providers if you wanted to make an exception, but as VPNs are sometimes used for scraping too, I can see why you might not want that.

              • melroy@kbin.melroy.org
                link
                fedilink
                arrow-up
                1
                ·
                4 hours ago

                Yea so the same servers and IP addresses VPN provider use (which are again just cloud providers) are also used for other purposes mainly by scrapers and DDos attacks indeed. Too bad all those cloud providers do not act accordingly in the past 5 - 10 years. Since its getting worse and worse. With the increase of datacenters and centralization, I consider it a duty of these companies to take action to stop these scammers, spammers, scrapers, and attackers.

                However, that is often not the case. Now I must say, I created Angie Guardian myself (alternative to Anubis). So hopefully soon I can slowly open some of ASN bans. And see how it goes.