Files
cowrie/bin/createfs
Florian Pelgrim 6c7417393a Flake8 fixes (#857)
* Adding flake8-import-order to linters

We need some kind of linter to enforce the code style.
Doing this by hand is error prone and no one will really watch for it.

I decieded to use the default style since it's from my point of view the
best to read and more strictes version.

* Enforcing imports order on bin/

* Enforcing imports order on src/twisted/

* Enforcing imports order on setup.py

* Enforcing imports order on src/cowrie/commands

* Enforcing imports order on src/cowrie/core

* Enforcing imports order on src/cowrie/dblog

* Enforcing imports order on src/cowrie/insults

* Enforcing imports order on src/cowrie/output

* Enforcing imports order on src/cowrie/proxy

* Enforcing imports order on src/cowrie/python

* Enforcing imports order on src/cowrie/shell

* Enforcing imports order on src/cowrie/ssh

* Enforcing imports order on src/cowrie/telnet

* Enforcing imports order on src/cowrie/test

* Reformat file, improved readability

* flake8 E5,E701 src/cowrie/commands/base.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/base.py
src/cowrie/commands/base.py:47:121: E501 line too long (180 > 120 characters)
src/cowrie/commands/base.py:48:121: E501 line too long (182 > 120 characters)
src/cowrie/commands/base.py:49:121: E501 line too long (149 > 120 characters)
src/cowrie/commands/base.py:50:121: E501 line too long (175 > 120 characters)
src/cowrie/commands/base.py:52:121: E501 line too long (127 > 120 characters)
src/cowrie/commands/base.py:54:121: E501 line too long (185 > 120 characters)
src/cowrie/commands/base.py:56:121: E501 line too long (130 > 120 characters)
src/cowrie/commands/base.py:57:121: E501 line too long (122 > 120 characters)
src/cowrie/commands/base.py:59:121: E501 line too long (188 > 120 characters)
src/cowrie/commands/base.py:60:121: E501 line too long (188 > 120 characters)
src/cowrie/commands/base.py:61:121: E501 line too long (145 > 120 characters)
src/cowrie/commands/base.py:63:121: E501 line too long (142 > 120 characters)
src/cowrie/commands/base.py:64:121: E501 line too long (152 > 120 characters)
src/cowrie/commands/base.py:66:121: E501 line too long (128 > 120 characters)
src/cowrie/commands/base.py:67:121: E501 line too long (123 > 120 characters)
src/cowrie/commands/base.py:72:121: E501 line too long (130 > 120 characters)
src/cowrie/commands/base.py:74:121: E501 line too long (125 > 120 characters)
src/cowrie/commands/base.py:75:121: E501 line too long (140 > 120 characters)
src/cowrie/commands/base.py:76:121: E501 line too long (133 > 120 characters)
src/cowrie/commands/base.py:78:121: E501 line too long (124 > 120 characters)
src/cowrie/commands/base.py:79:121: E501 line too long (122 > 120 characters)
src/cowrie/commands/base.py:80:121: E501 line too long (129 > 120 characters)
src/cowrie/commands/base.py:81:121: E501 line too long (150 > 120 characters)
src/cowrie/commands/base.py:83:121: E501 line too long (129 > 120 characters)
src/cowrie/commands/base.py:544:68: E502 the backslash is redundant between brackets
src/cowrie/commands/base.py:553:68: E502 the backslash is redundant between brackets
src/cowrie/commands/base.py:582:66: E502 the backslash is redundant between brackets
src/cowrie/commands/base.py:696:78: E502 the backslash is redundant between brackets
24    E501 line too long (180 > 120 characters)
4     E502 the backslash is redundant between brackets
28

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/commands/curl.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/curl.py
src/cowrie/commands/curl.py:413:106: E502 the backslash is redundant between brackets
1     E502 the backslash is redundant between brackets
1

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/commands/fs.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/fs.py
src/cowrie/commands/fs.py:438:74: E502 the backslash is redundant between brackets
src/cowrie/commands/fs.py:465:67: E502 the backslash is redundant between brackets
2     E502 the backslash is redundant between brackets
2

* flake8 E5, E701 src/cowrie/commands/gcc.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/gcc.py
src/cowrie/commands/gcc.py:167:121: E501 line too long (635 > 120 characters)
1     E501 line too long (635 > 120 characters)
1

* flake8 E5, E701 src/cowrie/commands/ifconfig.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/ifconfig.py
src/cowrie/commands/ifconfig.py:11:121: E501 line too long (145 > 120 characters)
src/cowrie/commands/ifconfig.py:13:121: E501 line too long (138 > 120 characters)
2     E501 line too long (145 > 120 characters)
2

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/commands/netstat.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/netstat.py
src/cowrie/commands/netstat.py:21:121: E501 line too long (126 > 120 characters)
1     E501 line too long (126 > 120 characters)
1

* flake8 E5, E701 src/cowrie/commands/ping.py

flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/ping.py
src/cowrie/commands/ping.py:81:74: E502 the backslash is redundant between brackets
1     E502 the backslash is redundant between brackets
1

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/commands/ssh.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/ssh.py
src/cowrie/commands/ssh.py:79:87: E502 the backslash is redundant between brackets
src/cowrie/commands/ssh.py:99:81: E502 the backslash is redundant between brackets
2     E502 the backslash is redundant between brackets
2

* flake8 E5, E701 src/cowrie/commands/sudo.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/sudo.py
src/cowrie/commands/sudo.py:14:121: E501 line too long (133 > 120 characters)
src/cowrie/commands/sudo.py:15:121: E501 line too long (166 > 120 characters)
src/cowrie/commands/sudo.py:16:121: E501 line too long (122 > 120 characters)
src/cowrie/commands/sudo.py:24:121: E501 line too long (133 > 120 characters)
src/cowrie/commands/sudo.py:25:121: E501 line too long (166 > 120 characters)
src/cowrie/commands/sudo.py:26:121: E501 line too long (122 > 120 characters)
6     E501 line too long (133 > 120 characters)
6

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/commands/wget.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/wget.py
src/cowrie/commands/wget.py:50:19: E701 multiple statements on one line (colon)
src/cowrie/commands/wget.py:264:69: E502 the backslash is redundant between brackets
1     E502 the backslash is redundant between brackets
1     E701 multiple statements on one line (colon)
2

* flake8 E5, E701 src/cowrie/commands/yum.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/commands/yum.py
src/cowrie/commands/yum.py:8:1: F401 'datetime.datetime' imported but unused
src/cowrie/commands/yum.py:223:121: E501 line too long (193 > 120 characters)
src/cowrie/commands/yum.py:228:121: E501 line too long (193 > 120 characters)
src/cowrie/commands/yum.py:237:121: E501 line too long (193 > 120 characters)
3     E501 line too long (193 > 120 characters)
1     F401 'datetime.datetime' imported but unused
4

* flake8 E5, E701 src/cowrie/core/cef.py

flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/core
src/cowrie/core/cef.py:53:121: E501 line too long (122 > 120 characters)
1     E501 line too long (122 > 120 characters)
1

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/output/hpfeeds.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/output/hpfeeds.py
src/cowrie/output/hpfeeds.py:61:26: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:80:29: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:147:22: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:166:22: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:173:26: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:180:30: E701 multiple statements on one line (colon)
src/cowrie/output/hpfeeds.py:186:34: E701 multiple statements on one line (colon)
7     E701 multiple statements on one line (colon)
7

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/output/sqlite.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/output/sqlite.py
src/cowrie/output/sqlite.py:73:62: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:74:60: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:82:62: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:83:60: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:91:41: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:92:67: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py💯41: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:101:67: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:109:62: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:116:45: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:117:75: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:125:45: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:126:75: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:134:41: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:135:65: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:143:67: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:149:69: E502 the backslash is redundant between brackets
src/cowrie/output/sqlite.py:165:57: E502 the backslash is redundant between brackets
18    E502 the backslash is redundant between brackets
18

* flake8 E5, E701 src/cowrie/output/mysql.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/output/mysql.py
src/cowrie/output/mysql.py:108:77: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:109:71: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:115:75: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:116:73: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:122:75: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:127:54: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:128:80: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:133:54: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:134:80: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:139:58: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:140:88: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:147:58: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:148:88: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:154:58: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:155:88: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:161:54: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:162:78: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:169:68: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:175:70: E502 the backslash is redundant between brackets
src/cowrie/output/mysql.py:191:73: E502 the backslash is redundant between brackets
20    E502 the backslash is redundant between brackets
20

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* flake8 E5, E701 src/cowrie/shell/shlex.py

Before
flake8 --count --select=E1,E2,E3,E5,E701,E901,E999,F401,F821,F822,F823 --max-line-length=120 --statistics src/cowrie/shell
src/cowrie/shell/protocol.py:37:121: E501 line too long (144 > 120 characters)
src/cowrie/shell/shlex.py:111:51: E502 the backslash is redundant between brackets
1     E501 line too long (144 > 120 characters)
1     E502 the backslash is redundant between brackets
2

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Reformat code, improved readability

* Update linters to check for E5 and E701

* Update travis file to the current state

- Removed every checker we do not use. That means twistedchecker
and pytest.
- Updated the flake8 command to match our current clean state
- Removed unused old code

* Fix format typo of cowrie.direct-tcpip.data log event

* Update travis to match the current state

Travis checks now every fixed flake8 error.
Also I removed unused code from it to keep it clean.

* Removed vim file markers

We don't want to have editor specific code in our source files.
Fix your IDE but don't do it in the code.

* src/cowrie/test/test_echo.py removed trailing whitespace

* src/cowrie/commands/yum.py removed trailing whitespace

* src/cowrie/commands/netstat.py removed trailing whitespace

* Checking for flake8 W291, W293

* flake8 F811 ./src/cowrie/commands/scp.py

I made sure to delete only the oldest version of the code.

Before
flake8 --count --select=F811 --application-import-names cowrie --max-line-length=120 --statistics .
./src/cowrie/commands/scp.py:209:5: F811 redefinition of unused 'handle_CTRL_D' from line 192
1     F811 redefinition of unused 'handle_CTRL_D' from line 192
1

* Update flake8 checks

* Fix flake8 E722, F841 src/cowrie/commands/wget.py

* Fix flake8 E722 src/cowrie/commands/ping.py

* Fix flake8 E722,F841 src/cowrie/commands/nc.py

* Fix flake8 E722, E741, F841 src/cowrie/commands/base.py

* Flake8 E731 src/cowrie/commands/base.py

I removed all lambda functions and replaced them with proper code.
That means straight function calls or not calling anything because it's
not needed.

* Fix flake8 E722, F841 src/cowrie/commands/scp.py

* Fix flake8 E722, F841 src/cowrie/commands/ssh.py

* Fix flake8 E712 src/cowrie/commands/iptables.py

* Fix flake8 E741 src/cowrie/commands/adduser.py

* Fix flake8 F841 src/cowrie/commands/tftp.py

* Fix flake8 E722 in all files

* Fix flake8 E711 bin/createdynamicprocess.py

* Fix flake8 E712, E713 src/cowrie/core/dblog.py

* Fix flake8 E712, E713 src/cowrie/shell/avatar.py

* Fix flake8 E712 src/cowrie/shell/fs.py

* Fix flake8 E712, E741 src/cowrie/commands/ls.py

* Fix flake8 E712, E713 src/cowrie/dblog/xmpp.py

* Fix flake8 E731, F841 src/cowrie/commands/gcc.py

* Fix flake8 F841 src/cowrie/insults/insults.py

* Fix flake8 F841 src/cowrie/telnet/session.py

* Fix flake8 F841 src/cowrie/commands/fs.py

* Fix flake8 E741,F841 src/cowrie/commands/last.py

* Fix flake8 E741 src/cowrie/commands/netstat.py

* Fix flake8 F841 src/cowrie/commands/free.py

* Fix flake8 F841 src/cowrie/commands/sudo.py

* Fix flake8 F841 src/cowrie/commands/curl.py

* Fix flake8 F841 src/cowrie/commands/base64.py

* Fix flake8 F841 src/cowrie/commands/service.py

* Fix flake8 F841 src/cowrie/ssh/factory.py

* Fix flake8 F841 src/cowrie/ssh/transport.py

* Fix flake8 F841 src/cowrie/output/csirtg.py

* Fix flake8 F841 src/cowrie/output/kafka.py

* Fix flake8 F841 src/cowrie/output/mongodb.py

* Fix flake8 F741 src/cowrie/output/hpfeeds.py

* Fix flake8 F741 src/cowrie/output/hpfeeds.py

* Fix flake8 F841 src/proxy/session.py

* Fix flake8 F841 src/cowrie/core/utils.py

* Update travis and Dockerfile for flake8 checks
2018-08-18 15:52:45 +04:00

159 lines
4.3 KiB
Python
Executable File

#!/usr/bin/env python
###############################################################
# This program creates a cowrie file system pickle file.
#
# This is meant to build a brand new filesystem.
# To edit the file structure, please use 'bin/fsctl'
#
##############################################################
import fnmatch
import getopt
import os
import pickle
import sys
from stat import *
A_NAME, A_TYPE, A_UID, A_GID, A_SIZE, A_MODE, \
A_CTIME, A_CONTENTS, A_TARGET, A_REALFILE = range(0, 10)
T_LINK, T_DIR, T_FILE, T_BLK, T_CHR, T_SOCK, T_FIFO = range(0, 7)
PROC = False
VERBOSE = False
blacklist_files = [
'/root/fs.pickle',
'/root/createfs',
'*cowrie*',
'*kippo*',
]
def logit(ftxt):
if VERBOSE:
sys.stderr.write(ftxt)
def checkblacklist(ftxt):
for value in blacklist_files:
if fnmatch.fnmatch(ftxt, value):
return True
return False
def recurse(localroot, root, tree, maxdepth=100):
if maxdepth == 0: return
localpath = os.path.join(localroot, root[1:])
logit(' %s\n' % (localpath))
if not os.access(localpath, os.R_OK):
logit(' Cannot access %s\n' % localpath)
return
for name in os.listdir(localpath):
fspath = os.path.join(root, name)
if checkblacklist(fspath):
continue
path = os.path.join(localpath, name)
try:
if os.path.islink(path):
s = os.lstat(path)
else:
s = os.stat(path)
except OSError:
continue
entry = [name, T_FILE, s.st_uid, s.st_gid, s.st_size, s.st_mode, \
int(s.st_mtime), [], None, None]
if S_ISLNK(s[ST_MODE]):
if not os.access(path, os.R_OK):
logit(' Cannot access link: %s\n' % path)
continue
realpath = os.path.realpath(path)
if not realpath.startswith(localroot):
logit(' Link "%s" has real path "%s" outside local root "%s"\n' \
% (path, realpath, localroot))
continue
else:
entry[A_TYPE] = T_LINK
entry[A_TARGET] = realpath[len(localroot):]
elif S_ISDIR(s[ST_MODE]):
entry[A_TYPE] = T_DIR
if (PROC or not localpath.startswith('/proc/')) and maxdepth > 0:
recurse(localroot, fspath, entry[A_CONTENTS], maxdepth - 1)
elif S_ISREG(s[ST_MODE]):
entry[A_TYPE] = T_FILE
elif S_ISBLK(s[ST_MODE]):
entry[A_TYPE] = T_BLK
elif S_ISCHR(s[ST_MODE]):
entry[A_TYPE] = T_CHR
elif S_ISSOCK(s[ST_MODE]):
entry[A_TYPE] = T_SOCK
elif S_ISFIFO(s[ST_MODE]):
entry[A_TYPE] = T_FIFO
else:
sys.stderr.write('We should handle %s' % path)
sys.exit(1)
tree.append(entry)
def help(brief=False):
print('Usage: %s [-h] [-v] [-p] [-l dir] [-d maxdepth] [-o file]\n' % \
os.path.basename(sys.argv[0]))
if not brief:
print(' -v verbose')
print(' -p include /proc')
print(' -l <dir> local root directory (default is current working directory)')
print(' -d <depth> maximum depth (default is full depth)')
print(' -o <file> write output to file instead of stdout')
print(' -h display this help\n')
sys.exit(1)
if __name__ == '__main__':
maxdepth = 100
localroot = os.getcwd()
output = ''
try:
optlist, args = getopt.getopt(sys.argv[1:], 'hvpl:d:o:', ['help'])
except getopt.GetoptError as error:
sys.stderr.write('Error: %s\n' % error)
help()
for o, a in optlist:
if o == '-v':
VERBOSE = True
elif o == '-p':
PROC = True
elif o == '-l':
localroot = a
elif o == '-d':
maxdepth = int(a)
elif o == '-o':
output = a
elif o in ['-h', '--help']:
help()
if output and os.path.isfile(output):
sys.stderr.write('File: %s exists!\n' % output)
sys.exit(1)
logit('Processing:\n')
tree = ['/', T_DIR, 0, 0, 0, 0, 0, [], '']
recurse(localroot, '/', tree[A_CONTENTS], maxdepth)
if output:
pickle.dump(tree, open(output, 'wb'))
else:
print(pickle.dumps(tree))