croc: Easily and Securely Send Files Between Computers
croc is a powerful Go library that enables secure, peer-to-peer file transfers between computers without requiring any central server or cloud storage. With over 34,286 stars on GitHub, this library has become the go-to solution for developers who need to implement secure file sharing functionality in their applications.
The library solves a fundamental problem in distributed systems: how to reliably transfer files between machines while maintaining end-to-end encryption and without relying on third-party services. Whether you’re building a file-sharing application, implementing backup systems, or creating distributed applications, croc provides the building blocks for secure, efficient file transfers.
Real-world use cases include:
- Securely sending large files between development environments
- Implementing backup and restore functionality in distributed systems
- Creating peer-to-peer file sharing applications
- Transferring configuration files and sensitive data between servers
- Building collaborative tools that require file exchange
Key Features
End-to-End Encryption: croc uses PAKE (Password-Authenticated Key Exchange) to establish secure connections. This means files are encrypted before leaving the sender’s machine and only decrypted by the intended recipient, with no intermediary able to access the data.
Relay Support: When direct peer-to-peer connections aren’t possible (due to NATs or firewalls), croc automatically falls back to using relay servers. This ensures file transfers work in any network environment while maintaining security.
Resumable Transfers: Large file transfers can be interrupted and resumed later without starting over. This is crucial for transferring files over unreliable networks or when dealing with very large datasets.
Compression and Deduplication: The library automatically compresses files during transfer and detects duplicate data, significantly reducing transfer times and bandwidth usage.
Cross-Platform Compatibility: croc works seamlessly across different operating systems and architectures, making it ideal for heterogeneous environments.
Progress Tracking: Built-in progress reporting allows applications to provide real-time feedback to users about transfer status, speed, and estimated completion time.
Customizable Security: Developers can configure security parameters like encryption strength, authentication methods, and relay server selection based on their specific requirements.
Installation and Setup
To install croc, you’ll need Go 1.16 or later. The installation is straightforward:
1go get github.com/schollz/croc/v9
For the latest version with all features:
1go install github.com/sollz/croc/v9@latest
Version Requirements:
- Go 1.16+
- No additional dependencies required
Verification:
1package main
2
3import (
4 "fmt"
5 "github.com/schollz/croc/v9"
6)
7
8func main() {
9 fmt.Println("croc version:", croc.Version())
10}
Expected output:
1croc version: v9.x.x
Basic Usage
Here’s a minimal example demonstrating the core functionality of croc:
1package main
2
3import (
4 "context"
5 "fmt"
6 "github.com/schollz/croc/v9"
7 "log"
8)
9
10func main() {
11 // Create a new croc session
12 session, err := croc.New()
13 if err != nil {
14 log.Fatal(err)
15 }
16
17 // Set up sender
18 sender := croc.NewSender(session)
19 err = sender.Send(context.Background(), "example.txt")
20 if err != nil {
21 log.Fatal(err)
22 }
23
24 // Get the code for sharing
25 code, err := sender.Code()
26 if err != nil {
27 log.Fatal(err)
28 }
29
30 fmt.Printf("Send code: %s\n", code)
31}
Expected output:
1Send code: fox-dragon-apple
This simple example creates a croc session, sends a file, and generates a shareable code. The recipient can use this code to receive the file.
Real-World Examples
Example 1: Production File Transfer Service
1package main
2
3import (
4 "context"
5 "crypto/rand"
6 "encoding/json"
7 "fmt"
8 "github.com/schollz/croc/v9"
9 "github.com/schollz/croc/v9/relay"
10 "log"
11 "net/http"
12 "os"
13 "path/filepath"
14 "sync"
15 "time"
16)
17
18type FileTransferService struct {
19 crocSession *croc.Croc
20 relayServer *relay.Server
21 mu sync.RWMutex
22 transfers map[string]*TransferStatus
23}
24
25type TransferStatus struct {
26 File string `json:"file"`
27 Status string `json:"status"`
28 Progress float64 `json:"progress"`
29 StartTime time.Time `json:"start_time"`
30 EndTime time.Time `json:"end_time,omitempty"`
31 Error string `json:"error,omitempty"`
32}
33
34func NewFileTransferService() (*FileTransferService, error) {
35 crocSession, err := croc.New(
36 croc.WithRelayEnabled(true),
37 croc.WithCompression(true),
38 croc.WithEncryptionStrength(256),
39 )
40 if err != nil {
41 return nil, err
42 }
43
44 relayServer, err := relay.NewServer(":9009")
45 if err != nil {
46 return nil, err
47 }
48
49 service := &FileTransferService{
50 crocSession: crocSession,
51 relayServer: relayServer,
52 transfers: make(map[string]*TransferStatus),
53 }
54
55 return service, nil
56}
57
58func (s *FileTransferService) Start() error {
59 go s.relayServer.Start()
60 return nil
61}
62
63func (s *FileTransferService) Stop() error {
64 s.relayServer.Stop()
65 return nil
66}
67
68func (s *FileTransferService) SendFile(ctx context.Context, filePath string) (string, error) {
69 // Validate file
70 fileInfo, err := os.Stat(filePath)
71 if err != nil {
72 return "", fmt.Errorf("file not found: %v", err)
73 }
74
75 if fileInfo.IsDir() {
76 return "", fmt.Errorf("directories not supported in this example")
77 }
78
79 if fileInfo.Size() == 0 {
80 return "", fmt.Errorf("empty files cannot be transferred")
81 }
82
83 // Generate transfer ID
84 transferID := generateTransferID()
85
86 // Create sender
87 sender, err := croc.NewSender(s.crocSession)
88 if err != nil {
89 return "", err
90 }
91
92 // Set up progress tracking
93 status := &TransferStatus{
94 File: filePath,
95 Status: "pending",
96 Progress: 0,
97 StartTime: time.Now(),
98 }
99
100 s.mu.Lock()
101 s.transfers[transferID] = status
102 s.mu.Unlock()
103
104 // Start transfer with progress callback
105 err = sender.Send(ctx, filePath, croc.WithProgress(func(sent, total int64) {
106 s.mu.Lock()
107 status.Progress = float64(sent) / float64(total) * 100
108 s.mu.Unlock()
109 }))
110
111 if err != nil {
112 s.mu.Lock()
113 status.Status = "failed"
114 status.Error = err.Error()
115 status.EndTime = time.Now()
116 s.mu.Unlock()
117 return "", err
118 }
119
120 // Get code and update status
121 code, err := sender.Code()
122 if err != nil {
123 s.mu.Lock()
124 status.Status = "failed"
125 status.Error = err.Error()
126 status.EndTime = time.Now()
127 s.mu.Unlock()
128 return "", err
129 }
130
131 s.mu.Lock()
132 status.Status = "completed"
133 status.EndTime = time.Now()
134 s.mu.Unlock()
135
136 return code, nil
137}
138
139func (s *FileTransferService) GetStatus(transferID string) (*TransferStatus, error) {
140 s.mu.RLock()
141 defer s.mu.RUnlock()
142 status, exists := s.transfers[transferID]
143 if !exists {
144 return nil, fmt.Errorf("transfer not found")
145 }
146 return status, nil
147}
148
149func generateTransferID() string {
150 b := make([]byte, 16)
151 _, err := rand.Read(b)
152 if err != nil {
153 return fmt.Sprintf("transfer-%d", time.Now().UnixNano())
154 }
155 return fmt.Sprintf("%x", b)
156}
157
158func main() {
159 service, err := NewFileTransferService()
160 if err != nil {
161 log.Fatal(err)
162 }
163
164 defer service.Stop()
165
166 err = service.Start()
167 if err != nil {
168 log.Fatal(err)
169 }
170
171 // Example usage
172 code, err := service.SendFile(context.Background(), "large-file.zip")
173 if err != nil {
174 log.Fatal(err)
175 }
176
177 fmt.Printf("Transfer code: %s\n", code)
178
179 // Wait for completion
180 time.Sleep(5 * time.Second)
181
182 // Get status
183 status, err := service.GetStatus(code)
184 if err != nil {
185 log.Fatal(err)
186 }
187
188 statusJSON, _ := json.MarshalIndent(status, "", " ")
189 fmt.Println(string(statusJSON))
190}
Example 2: Distributed Backup System
1package main
2
3import (
4 "context"
5 "crypto/sha256"
6 "fmt"
7 "github.com/schollz/croc/v9"
8 "github.com/schollz/croc/v9/relay"
9 "log"
10 "os"
11 "path/filepath"
12 "sync"
13 "time"
14)
15
16type BackupNode struct {
17 crocSession *croc.Croc
18 nodeID string
19 backupDir string
20 mu sync.RWMutex
21 backups map[string]*BackupRecord
22}
23
24type BackupRecord struct {
25 File string `json:"file"`
26 Hash string `json:"hash"`
27 Size int64 `json:"size"`
28 Timestamp time.Time `json:"timestamp"`
29 Status string `json:"status"`
30}
31
32func NewBackupNode(nodeID, backupDir string) (*BackupNode, error) {
33 crocSession, err := croc.New(
34 croc.WithRelayEnabled(true),
35 croc.WithCompression(true),
36 croc.WithEncryptionStrength(256),
37 croc.WithTimeout(30*time.Minute),
38 )
39 if err != nil {
40 return nil, err
41 }
42
43 return &BackupNode{
44 crocSession: crocSession,
45 nodeID: nodeID,
46 backupDir: backupDir,
47 backups: make(map[string]*BackupRecord),
48 }, nil
49}
50
51func (n *BackupNode) CreateBackup(ctx context.Context, sourcePath string) error {
52 // Calculate file hash
53 fileHash, err := calculateFileHash(sourcePath)
54 if err != nil {
55 return fmt.Errorf("hash calculation failed: %v", err)
56 }
57
58 // Generate backup filename
59 backupFilename := fmt.Sprintf("%s-%x.backup", filepath.Base(sourcePath), fileHash[:8])
60 backupPath := filepath.Join(n.backupDir, backupFilename)
61
62 // Check if backup already exists
63 n.mu.RLock()
64 if record, exists := n.backups[backupFilename]; exists {
65 if record.Hash == fmt.Sprintf("%x", fileHash) {
66 n.mu.RUnlock()
67 log.Printf("Backup already exists: %s", backupFilename)
68 return nil
69 }
70 }
71 n.mu.RUnlock()
72
73 // Create sender
74 sender, err := croc.NewSender(n.crocSession)
75 if err != nil {
76 return err
77 }
78
79 // Start backup transfer
80 err = sender.Send(ctx, sourcePath, croc.WithProgress(func(sent, total int64) {
81 log.Printf("[%s] Backup progress: %.2f%%", n.nodeID, float64(sent)/float64(total)*100)
82 }))
83
84 if err != nil {
85 return fmt.Errorf("backup failed: %v", err)
86 }
87
88 // Get transfer code
89 code, err := sender.Code()
90 if err != nil {
91 return fmt.Errorf("code generation failed: %v", err)
92 }
93
94 // Update backup record
95 n.mu.Lock()
96 n.backups[backupFilename] = &BackupRecord{
97 File: backupFilename,
98 Hash: fmt.Sprintf("%x", fileHash),
99 Size: getFileSize(sourcePath),
100 Timestamp: time.Now(),
101 Status: "completed",
102 }
103 n.mu.Unlock()
104
105 log.Printf("[%s] Backup completed: %s (code: %s)", n.nodeID, backupFilename, code)
106 return nil
107}
108
109func (n *BackupNode) RestoreBackup(ctx context.Context, backupFilename, destinationPath string) error {
110 n.mu.RLock()
111 record, exists := n.backups[backupFilename]
112 n.mu.RUnlock()
113
114 if !exists {
115 return fmt.Errorf("backup not found: %s", backupFilename)
116 }
117
118 // Create receiver
119 receiver, err := croc.NewReceiver(n.crocSession)
120 if err != nil {
121 return err
122 }
123
124 // Start restore
125 err = receiver.Receive(ctx, destinationPath, croc.WithCode(record.File))
126 if err != nil {
127 return fmt.Errorf("restore failed: %v", err)
128 }
129
130 log.Printf("[%s] Restore completed: %s", n.nodeID, backupFilename)
131 return nil
132}
133
134func calculateFileHash(filePath string) ([]byte, error) {
135 file, err := os.Open(filePath)
136 if err != nil {
137 return nil, err
138 }
139 defer file.Close()
140
141 hasher := sha256.New()
142 if _, err := io.Copy(hasher, file); err != nil {
143 return nil, err
144 }
145
146 return hasher.Sum(nil), nil
147}
148
149func getFileSize(filePath string) int64 {
150 fileInfo, err := os.Stat(filePath)
151 if err != nil {
152 return 0
153 }
154 return fileInfo.Size()
155}
156
157func main() {
158 node, err := NewBackupNode("node-1", "./backups")
159 if err != nil {
160 log.Fatal(err)
161 }
162
163 // Create backup directory
164 os.MkdirAll(node.backupDir, 0755)
165
166 // Example backup
167 err = node.CreateBackup(context.Background(), "important-data.txt")
168 if err != nil {
169 log.Fatal(err)
170 }
171
172 // List backups
173 node.mu.RLock()
174 for filename, record := range node.backups {
175 fmt.Printf("Backup: %s\n Hash: %s\n Size: %d bytes\n Time: %s\n Status: %s\n",
176 filename, record.Hash, record.Size, record.Timestamp, record.Status)
177 }
178 node.mu.RUnlock()
179}
Best Practices and Common Pitfalls
Always handle context cancellation: File transfers can take a long time, so always use context.Context to allow for cancellation and timeouts. Never start transfers without proper cancellation support.
Validate file paths and permissions: Before starting transfers, verify that source files exist and are readable, and that destination directories are writable. This prevents runtime errors and improves user experience.
Implement proper error handling: croc returns detailed error messages. Always check and log these errors appropriately. Common errors include network timeouts, authentication failures, and file permission issues.
Use appropriate timeouts: Set realistic timeouts based on file sizes and network conditions. For large files, use longer timeouts (30+ minutes), while small files can use shorter ones.
Monitor progress and implement retries: For production systems, implement progress tracking and automatic retry logic for failed transfers. This improves reliability in unstable network conditions.
Secure your relay servers: If using relay servers, ensure they’re properly secured and monitored. Consider implementing authentication and rate limiting for relay endpoints.
Common pitfalls to avoid:
- Forgetting to close file handles after transfers
- Not handling network interruptions gracefully
- Using weak encryption settings in production
- Ignoring progress callbacks, leading to poor user experience
- Failing to clean up temporary files after transfers
Conclusion
croc is an exceptional Go library that simplifies secure file transfers between computers. With its robust encryption, automatic relay support, and comprehensive feature set, it’s an invaluable tool for developers building distributed systems, file-sharing applications, or any software that requires secure data exchange.
The library’s 34,286 GitHub stars reflect its popularity and reliability in the Go ecosystem. Whether you’re building a simple file transfer utility or a complex distributed backup system, croc provides the building blocks you need with minimal complexity.
Key takeaways:
- End-to-end encryption ensures data security
- Automatic relay support handles network restrictions
- Resumable transfers improve reliability
- Cross-platform compatibility simplifies deployment
- Comprehensive API supports production use cases
GitHub Repository: https://github.com/schollz/croc
Documentation: https://pkg.go.dev/github.com/schollz/croc/v9
Start using croc today to add secure, reliable file transfer capabilities to your Go applications!