Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion lib/ood_core/job/adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,15 @@ def cluster_info
#
# For certain adapters this may speed up the response since
# adapters can get by without populating the entire Info object
#
# @param host [#to_s, nil] defaults to nil (and all hosts are queried)
# Restricts the listing to the specified host. Only meaningful for adapters
# that manage jobs across multiple hosts, like LinuxHost or Systemd. Adapters
# that talk to a single scheduler will accept and ignore it. Adapters
# that honor it will raise unless the specified host is included in
# the ssh_hosts adapter configuration.
# @return [Array<Info>] information describing submitted jobs
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
raise NotImplementedError, "subclass did not define #info_all"
end

Expand Down
20 changes: 1 addition & 19 deletions lib/ood_core/job/adapters/ccq.rb
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])

# Retrieve info for all jobs from the resource manager
# @return [Array<Info>] information describing submitted jobs
def info_all(attrs: nil)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parent class doesn't have the host keyword. Not sure why it was implemented in linux_host - I can't find it being used in OnDemand.

def info_all(attrs: nil)
raise NotImplementedError, "subclass did not define #info_all"
end

If anything we can open the interface to attrs and **kwargs but it seems unnecessary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't scroll down far enough - you add it to the parent class.

In any case the comment I think still stands - open it up to **kwargs or remove it from the child clasess. I would probably vote for the latter as I don't see it actually being used in OnDemand.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I might have to sit with this for a bit as I'm not sure I like either route very much. It seems kinda wrong to rip the host filtering feature out of LinuxHost and Systemd altogether when it has been there for 7ish years, but if we aren't using it then maybe nobody would care. On the flip side, a blanket **kwargs feels like it adds margin for error here, when the intent of the ticket was making the adapter interface more consistent to reduce confusion caused downstream.

Looking back, it would have been nice if it had originally been added as a separate method, like info_where_host(, but this far along I bet its about the same impact to change it as it is to rip it out altogether, the impact being that older unmaintained stuff could break

def info_all(attrs: nil, host: nil)
args = []
args.concat ["-s", scheduler] unless scheduler.nil?

Expand Down Expand Up @@ -104,24 +104,6 @@ def status(id)
info(id).status
end

# This adapter does not implement hold and will always raise
# an exception.
# @param id [#to_s] the id of the job
# @raise [JobAdapterError] always
# @return [void]
def hold(_)
raise NotImplementedError, "subclass did not define #hold"
end

# This adapter does not implement release and will always raise
# an exception.
# @param id [#to_s] the id of the job
# @raise [JobAdapterError] always
# @return [void]
def release(_)
raise NotImplementedError, "subclass did not define #release"
end

# Delete the submitted job
# @param id [#to_s] the id of the job
# @return [void]
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/coder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
# For certain adapters this may speed up the response since
# adapters can get by without populating the entire Info object
# @return [Array<Info>] information describing submitted jobs
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
[]
end

Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/fujitsu_tcs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
# @see Adapter#info_all
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@fujitsu_tcs.get_jobs().map do |v|
parse_job_info(v)
end
Expand Down
4 changes: 2 additions & 2 deletions lib/ood_core/job/adapters/htcondor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ def initialize(opts = {})
# @raise [JobAdapterError] if something goes wrong submitting a job
# @return [String] the job id returned after successfully submitting a
# job
def submit(script)
def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
validate_native(script, Hash)
args = []
args.concat ["-batch-name", "#{script.job_name}"] unless script.job_name.nil?
Expand Down Expand Up @@ -432,7 +432,7 @@ def info(id)
# Retrieve information for all jobs
# @raise [JobAdapterError] if something goes wrong retrieving job info
# @return [Array<Info>] list of information describing submitted jobs
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
jobs = @htcondor.get_jobs
jobs.map { |job| parse_job_info(job) }
rescue Batch::Error => e
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/kubernetes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
# For certain adapters this may speed up the response since
# adapters can get by without populating the entire Info object
# @return [Array<Info>] information describing submitted jobs
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
batch.info_all(attrs: attrs)
rescue Batch::Error => e
raise JobAdapterError, e.message
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/linux_host.rb
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ def info_all(attrs: nil, host: nil)
# @param owner [#to_s, Array<#to_s>] the owner(s) of the jobs
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
def info_where_owner(_, attrs: nil)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't a linter say this is correct?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I could have gone either way on this, so I will update to not check names of positional args and revert this change

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe check with rubocop - I'm only saying that from memory. Maybe it'll say _owner is correct, I'm not 100% sure.

def info_where_owner(owner, attrs: nil)
info_all
end

Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/lsf.rb
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ def info(id)
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
# @see Adapter#info_all
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
batch.get_jobs.map { |v| info_for_batch_hash(v) }
rescue Batch::Error => e
raise JobAdapterError, e.message
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/pbspro.rb
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ def cluster_info
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
# @see Adapter#info_all
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@pbspro.get_jobs.map do |v|
parse_job_info(v)
end
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/psij.rb
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ def info(id)
end
end

def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@psij.get_jobs.map do |v|
parse_job_info(v)
end
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/sge.rb
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])

# Retrieve info for all jobs from the resource manager
# @return [Array<Info>] information describing submitted jobs
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@batch.get_all(owner: '*')
rescue Batch::Error => e
raise JobAdapterError, e.message
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/slurm.rb
Original file line number Diff line number Diff line change
Expand Up @@ -755,7 +755,7 @@ def accounts
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
# @see Adapter#info_all
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@slurm.get_jobs(attrs: attrs).map do |v|
parse_job_info(v)
end
Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/systemd.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def info_all(attrs: nil, host: nil)
# @param owner [#to_s, Array<#to_s>] the owner(s) of the jobs
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
def info_where_owner(_, attrs: nil)
def info_where_owner(owner, attrs: nil)
info_all
end

Expand Down
2 changes: 1 addition & 1 deletion lib/ood_core/job/adapters/torque.rb
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def submit(script, after: [], afterok: [], afternotok: [], afterany: [])
# @raise [JobAdapterError] if something goes wrong getting job info
# @return [Array<Info>] information describing submitted jobs
# @see Adapter#info_all
def info_all(attrs: nil)
def info_all(attrs: nil, host: nil)
@pbs.get_jobs.map do |k, v|
parse_job_info(k, v)
end
Expand Down
42 changes: 42 additions & 0 deletions test/job/adapter_test.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
require 'test_helper'
require 'ood_core/job/adapter.rb'

ADAPTER_FILES = Dir[File.join('ood_core', 'job', 'adapters', '*.rb')]
ADAPTER_FILES.each { |file| require file }

class AdapterTest < Minitest::Test
BASE = OodCore::Job::Adapter

# Every concrete class descending from the base adapter
def adapters
ObjectSpace.each_object(Class).select { |klass| klass < BASE }
end

# Public methods defined by the base class itself (not inherited from Object)
def base_methods
BASE.public_instance_methods(false)
end

def test_adapters_were_found
refute_empty adapters, 'No subclasses of OodCore::Job::Adapter were found'
end

def test_adapter_interfaces_match_base_class
adapters.each do |adapter|
base_methods.each do |name|
impl = adapter.instance_method(name)

# If the method is not overridden, the base class method runs
next if impl.owner == BASE

# If it is overridden, the signature must match the base exactly
expected = BASE.instance_method(name).parameters
actual = impl.parameters

assert_equal expected, actual,
"#{adapter}##{name} (defined in #{impl.owner}) has parameters " \
"#{actual.inspect}, but #{BASE}##{name} defines #{expected.inspect}"
end
end
end
end
Loading